-- Logs begin at Wed 2022-08-31 11:35:23 UTC, end at Wed 2022-08-31 13:08:16 UTC. -- Aug 31 12:03:50.414480 dv-cinder-855359-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 12:03:50.421881 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 12:03:50 2022] *** Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: nodename: dv-cinder-855359-1-iscsi Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: machine: x86_64 Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: clock source: unix Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: pcre jit disabled Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: detected number of CPU cores: 4 Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: current working directory: / Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: detected binary path: /usr/bin/uwsgi-core Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: your processes number limit is 47680 Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: your memory page size is 4096 bytes Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: detected max file descriptor number: 2048 Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: lock engine: pthread robust mutexes Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: thunder lock: enabled Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 12:03:50.427281 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 12:03:50.464846 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: Python main interpreter initialized at 0x555621c6d9c0 Aug 31 12:03:50.464846 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: python threads support enabled Aug 31 12:03:50.464846 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: your server socket listen backlog is limited to 100 connections Aug 31 12:03:50.464846 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: your mercy for graceful operations on workers is 90 seconds Aug 31 12:03:50.465245 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 12:03:50.465326 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: *** Operational MODE: preforking *** Aug 31 12:03:50.465562 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: *** uWSGI is running in multiple interpreter mode *** Aug 31 12:03:50.465562 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: spawned uWSGI master process (pid: 81473) Aug 31 12:03:50.465818 dv-cinder-855359-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 12:03:50.466580 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: spawned uWSGI worker 1 (pid: 81487, cores: 1) Aug 31 12:03:50.466694 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: spawned uWSGI worker 2 (pid: 81488, cores: 1) Aug 31 12:03:50.466850 dv-cinder-855359-1-iscsi devstack@keystone.service[81473]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 12:03:54.788864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 12:03:54.788864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: from cryptography.utils import int_from_bytes Aug 31 12:03:54.789710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 12:03:54.789710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: from cryptography.utils import int_from_bytes Aug 31 12:03:54.790143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 12:03:54.790143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: from cryptography.utils import int_from_bytes Aug 31 12:03:54.790250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 12:03:54.790250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: from cryptography.utils import int_from_bytes Aug 31 12:03:54.838619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 12:03:54.838831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 12:03:54.838942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] command line args: [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 12:03:54.839254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 12:03:54.839460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ================================================================================ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 12:03:54.839566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 12:03:54.839645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] admin_token = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.839714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 12:03:54.839843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] command line args: [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 12:03:54.839924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] config_dir = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.840104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.840223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 12:03:54.840311 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] config_source = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.840372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ================================================================================ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 12:03:54.840436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] debug = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.840498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] admin_token = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.840611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.840914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] config_dir = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.840914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] config_source = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] debug = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.841912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_config_append = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_dir = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_config_append = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.842886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_dir = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843830 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.843969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.844959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.845022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.845158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.845244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.845449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.845668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.845886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.846033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.846033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.846298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.846531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.846531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.846832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.846832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] publish_errors = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.847150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.847335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.847335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.847668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.847813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.848194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] publish_errors = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.848404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.849063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.849185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.849315 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.849456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] standard_threads = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.849580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.849580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.849837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.849949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.850059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.850177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] standard_threads = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.850288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.850288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_journal = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.850288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_json = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.850614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.850725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_stderr = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.850892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.851013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] use_syslog = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.851151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.851278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.851391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_journal = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.851391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_json = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.851625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.851744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_stderr = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.851862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] use_syslog = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.852402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.852524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 12:03:54.852648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.852760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.852876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.852992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.853103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.853219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.853338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.853450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.853560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.853703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.853859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.853972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.854093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.854206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.854315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.854425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.854806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.854924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855044 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.855973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.856089 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.856200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.856310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.856427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.856539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.856649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.856759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.856876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.857064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.857191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.857310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.857421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.857539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.857660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.857780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.857894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.858910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.859982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.860999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.861109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.861227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.861339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.861339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.861552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.861669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.861780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.861898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.862997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.863108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.863218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.863357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.863518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.864952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.865934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.866818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.867993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.868104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.868223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.868344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.868456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.868456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.868675 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.868792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.868904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.869902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.870014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.870124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.870235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.870345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.870447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.870564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.871954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.872950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873044 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.873932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.874993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.875087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.875180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.875314 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.875445 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.875559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.875672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.875784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.875910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.876946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.external = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.877059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.877169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.877280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.877418 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.877550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.877676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.877802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.877917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.password = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.token = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.external = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.878924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.879034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.879369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.password = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.879618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.880259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.token = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.880385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.880505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.880619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.880741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.880856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.880978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.881093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.881207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.881328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.881452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.881568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.881681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.881794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.881907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.882802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.883936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.884984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.885096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.885209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.885322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.885443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.885557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.885669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.885782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.885896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.886924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.887022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.887133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.887341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.887464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.887586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.887700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.887811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.887924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.888036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.888147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.888339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.888461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.888999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.888999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.889191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.889288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.889413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.889517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.889613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.889708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.889809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.889905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890202 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.890967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.891960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.892919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.893096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.893191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.893284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.893397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.893496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.893591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.893685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.893780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.894412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.894537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.894651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.894807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.894907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.895169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.895803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.895923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896261 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.896952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.897065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.897186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.897301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.897414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.897533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.897648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.897762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.897885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.898000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.898114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.898316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.898433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.898547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.898669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.898785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.898898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.899968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.900089 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.900204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.900317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.900430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.900543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.900656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.900777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.900893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.901920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.902956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.903069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.903182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.903327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.903442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.903442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.903670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.903670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.903886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.904918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.905901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906888 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.906888 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.907109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.907223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.907354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.907466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.907579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.907705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.907821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.907934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.908959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.909998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.910111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.910224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.910469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] role.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.911014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.911143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] role.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.911317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.911435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.911548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.911749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.911867 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.911981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.912096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.912210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.912322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.912445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.912600 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.912716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.912829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.912941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] role.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] role.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.913974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.914985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.915098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.915219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.915351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.915464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.915578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.915690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.915802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.915927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.916949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.917966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.918988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.919100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.919221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.919392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.919505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.919617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.919731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.919844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.919969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.920996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.921108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.921220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.921341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.921509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.921624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.921737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.921850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.921969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.922091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.922206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.922318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.922430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.922542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.922663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.922786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.922908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.923959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924307 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.924990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.925104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.925216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.925337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.925459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.925573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.925685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.925798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.925918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.926070 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.926339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.926550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.926550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.927057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.927183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.927311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.927425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.927539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.927652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.927764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.927891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.928226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.929150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.929150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.929150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 12:03:54.929150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 12:03:54.952697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.952926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.953152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.954785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.955018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.955257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.961744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.963893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.990480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.990682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.990806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:54.990968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.046065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.046252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.095589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.095804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.096827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.097071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.254383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.254593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.254821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.255041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.255263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.255358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.255484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.255566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.255713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.255954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.256176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.256390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.278984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.279197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.279303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.279395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.279462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.279608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.279692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.279834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.279937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.280060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.280241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.280241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.280482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.280482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.292006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.293201 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.305425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.305603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.305765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.306019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.306146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.306234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.306474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.306612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.306612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.306830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.306830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.307044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.307330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.307526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 12:03:55.324120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.324306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.324306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 12:03:55.324476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 12:03:55.324653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.324793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.324877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 12:03:55.324959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 12:03:55.325112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.325237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.325332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.325410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.325620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.325878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.326002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.326128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.335556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.335686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.335812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.335903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.336102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.336249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.336335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.336411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.343338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.343470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.346199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.346330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.346494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.346625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.346770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.346860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.347012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.347094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.347352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.347476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.347605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.347707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.348122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.348215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.348281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.348342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.348480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.348570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.348877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.349003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.349129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.349221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.372106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.372425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.373561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.373854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.373978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.374164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.378338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.378624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.378853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.378985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.379110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.379513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.385493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.385776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.385904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.386198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.389709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.390006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.390279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.390405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.390542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.390645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.390846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.390973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.391174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.391439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.391722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.391999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.392271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.392986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.393286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.393574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.409018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.409304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.410027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.410381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.413229 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.414472 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.416340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.416613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.417700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.417997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.420937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.422214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.424063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.425343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.427289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.427598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.427882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.428164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.428453 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.428719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.428824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.429024 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.429314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.429603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.429905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.430191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.444902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.445203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.445521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.445761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.446753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.447084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.447414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.447786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.452727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.454367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.455447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.455941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.456217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.456989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.458757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.458949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.459049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.459270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.459556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.459842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.460145 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.461704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.461992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.462399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.462689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.462988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.471398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.471683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.471963 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.472240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.472517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.472793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.473165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.474444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.474743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.475130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.475449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.475737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.476030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.476337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.488967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.492096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.492221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.494957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.495084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.496421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.497880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.498005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.498166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.498445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.499335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.500943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.501239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.501538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.505207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.506686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.508035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.509514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.509638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.509787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.510105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.510390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.512526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.512830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.513127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.513412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.518424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.518713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.518992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.521218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.521527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.521792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.525881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.526165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.526447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.526724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.526996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.527309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.527579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.527855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.528133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.528409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.528646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 12:03:55.528735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.528928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.529021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.529192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.529276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.529467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.529549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.529741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.530018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.530292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.530574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.530858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.531140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.531452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.531720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.531988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.532273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 12:03:55.563797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555621c6d9c0 pid: 81487 (default app) Aug 31 12:03:55.566844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555621c6d9c0 pid: 81488 (default app) Aug 31 12:03:55.567534 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0ae3-308e-40d2-9595-ed4ba14711a6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:03:55.567765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0ae3-308e-40d2-9595-ed4ba14711a6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:03:55.568036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0ae3-308e-40d2-9595-ed4ba14711a6 None None] PATH_INFO: `/v3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:03:55.572253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Wed Aug 31 12:03:55 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:02.220075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe3429-edd3-49c0-ac91-20be285acb46 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:02.220633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe3429-edd3-49c0-ac91-20be285acb46 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:02.220633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe3429-edd3-49c0-ac91-20be285acb46 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:02.225546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:04:02.232315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:02.233017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:02.233017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:02.302004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: INFO dbcounter [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] Registered counter for database keystone Aug 31 12:04:02.321550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG oslo_db.sqlalchemy.engines [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81487) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 12:04:02.329045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writer thread running {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 12:04:02.515307 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG passlib.handlers.bcrypt [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81487) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 12:04:02.515607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG passlib.handlers.bcrypt [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81487) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 12:04:02.576063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:02.577660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:02.711140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 12:04:03.174210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe3429-edd3-49c0-ac91-20be285acb46 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:03.247060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: INFO dbcounter [None req-3dfe3429-edd3-49c0-ac91-20be285acb46 None None] Registered counter for database keystone Aug 31 12:04:03.269410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG oslo_db.sqlalchemy.engines [None req-3dfe3429-edd3-49c0-ac91-20be285acb46 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81488) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 12:04:03.276678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writer thread running {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 12:04:03.404904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe3429-edd3-49c0-ac91-20be285acb46 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:03.409784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd3143-b24e-47dd-bb6d-24e597e9bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:03.410505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd3143-b24e-47dd-bb6d-24e597e9bf9c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:03.410799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd3143-b24e-47dd-bb6d-24e597e9bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:03.411059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd3143-b24e-47dd-bb6d-24e597e9bf9c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:03.419690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd3143-b24e-47dd-bb6d-24e597e9bf9c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:03.471840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING keystone.common.rbac_enforcer.enforcer [None req-edfd3143-b24e-47dd-bb6d-24e597e9bf9c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 12:04:03.472185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd3143-b24e-47dd-bb6d-24e597e9bf9c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:03.483154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Aug 31 12:04:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 12:04:03.489594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:03.504584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827bb89-67a5-459c-9692-664bd121eba9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:03.509052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4e750-5215-473f-8ce1-bbb696ff7e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:03.510188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4e750-5215-473f-8ce1-bbb696ff7e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:03.510467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4e750-5215-473f-8ce1-bbb696ff7e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:03.510752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4e750-5215-473f-8ce1-bbb696ff7e56 admin admin] PATH_INFO: `/v3/projects/b6117aaa4eb34295b2216e51a40eacde` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:03.512404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4e750-5215-473f-8ce1-bbb696ff7e56 admin admin] RBAC: Authorizing `identity:get_project(project_id=b6117aaa4eb34295b2216e51a40eacde)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:03.565698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4ef4e750-5215-473f-8ce1-bbb696ff7e56 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 12:04:03.566097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4e750-5215-473f-8ce1-bbb696ff7e56 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:03.570914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Wed Aug 31 12:04:03 2022] GET /identity/v3/projects/b6117aaa4eb34295b2216e51a40eacde => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:05.474780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e5fd5-db6d-4b10-befa-7e8e721b05e7 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:05.475360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e5fd5-db6d-4b10-befa-7e8e721b05e7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:05.475360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e5fd5-db6d-4b10-befa-7e8e721b05e7 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:05.479804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:05.485582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aaa73-9f78-40f1-866e-3086c9f001c2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:05.485981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aaa73-9f78-40f1-866e-3086c9f001c2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:05.486130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aaa73-9f78-40f1-866e-3086c9f001c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:05.507014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d96aaa73-9f78-40f1-866e-3086c9f001c2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:05.508596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d96aaa73-9f78-40f1-866e-3086c9f001c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:05.516891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 12:04:05.784902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e5fd5-db6d-4b10-befa-7e8e721b05e7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:05.799132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e5fd5-db6d-4b10-befa-7e8e721b05e7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:05.804877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0e812-6f38-4f4b-b4df-408bb7dc806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:05.805491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e812-6f38-4f4b-b4df-408bb7dc806f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:05.805766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e812-6f38-4f4b-b4df-408bb7dc806f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:05.806037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e812-6f38-4f4b-b4df-408bb7dc806f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:05.820629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0e812-6f38-4f4b-b4df-408bb7dc806f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:05.822457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0e812-6f38-4f4b-b4df-408bb7dc806f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:05.832210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Aug 31 12:04:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:04:05.838144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aaa73-9f78-40f1-866e-3086c9f001c2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:05.847622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aaa73-9f78-40f1-866e-3086c9f001c2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:05.852901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a162d-99d0-4462-ae41-00f2976cec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:05.853528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a162d-99d0-4462-ae41-00f2976cec7d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:05.853801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a162d-99d0-4462-ae41-00f2976cec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:05.854061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a162d-99d0-4462-ae41-00f2976cec7d admin admin] PATH_INFO: `/v3/users/186838d97c1140bb9bd1a7296d1414bf` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:05.855541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a162d-99d0-4462-ae41-00f2976cec7d admin admin] RBAC: Authorizing `identity:get_user(user_id=186838d97c1140bb9bd1a7296d1414bf)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:05.857414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a162d-99d0-4462-ae41-00f2976cec7d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:05.858987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Wed Aug 31 12:04:05 2022] GET /identity/v3/users/186838d97c1140bb9bd1a7296d1414bf => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:07.822833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab8b4-c412-413e-8d99-f3f0d1ab541f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:07.823467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab8b4-c412-413e-8d99-f3f0d1ab541f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:07.823467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab8b4-c412-413e-8d99-f3f0d1ab541f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:07.824735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:07.830232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab74f3-64e6-4ac9-9f8e-9ef7c0cdd2b2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:07.830750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab74f3-64e6-4ac9-9f8e-9ef7c0cdd2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:07.830750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab74f3-64e6-4ac9-9f8e-9ef7c0cdd2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:07.849660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-deab74f3-64e6-4ac9-9f8e-9ef7c0cdd2b2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:07.851008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-deab74f3-64e6-4ac9-9f8e-9ef7c0cdd2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:07.932146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1878c-c8fe-41d8-9230-4d2904314841 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:07.932429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1878c-c8fe-41d8-9230-4d2904314841 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:07.932710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1878c-c8fe-41d8-9230-4d2904314841 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:07.934043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:04:07.941200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:07.941200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:07.941200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:07.947400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:07.987891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG passlib.handlers.bcrypt [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81488) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 12:04:07.988282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG passlib.handlers.bcrypt [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81488) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 12:04:08.036877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:08.038507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:08.155004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:08.216778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab74f3-64e6-4ac9-9f8e-9ef7c0cdd2b2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:08.244922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:08.330971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab74f3-64e6-4ac9-9f8e-9ef7c0cdd2b2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:08.371621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d04a7-981b-4fc9-b56e-f9c957226f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:08.372269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d04a7-981b-4fc9-b56e-f9c957226f60 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:08.372552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d04a7-981b-4fc9-b56e-f9c957226f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:08.372826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d04a7-981b-4fc9-b56e-f9c957226f60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:08.385379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d04a7-981b-4fc9-b56e-f9c957226f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-968d04a7-981b-4fc9-b56e-f9c957226f60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:08.394661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:08.395942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:08.397283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:08.401883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:08.420635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d04a7-981b-4fc9-b56e-f9c957226f60 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:08.510623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa484794-7b17-4018-9e4d-2017c10f47a0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:08.513173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:08.515100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:08.538621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d04a7-981b-4fc9-b56e-f9c957226f60 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:08.555054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:08.555998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:08.556259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:08.556529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:08.557365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:08.557663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:08.559135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:08.572337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:08.574808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c854a14-97bb-4c9e-a88b-4a0af96eba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:08.575549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854a14-97bb-4c9e-a88b-4a0af96eba8f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:08.576420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854a14-97bb-4c9e-a88b-4a0af96eba8f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:08.576420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854a14-97bb-4c9e-a88b-4a0af96eba8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:08.577989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:08.650983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c854a14-97bb-4c9e-a88b-4a0af96eba8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:08.652527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c854a14-97bb-4c9e-a88b-4a0af96eba8f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:08.691949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ba92b-6417-475c-a3cd-fe32698a9c6b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:08.728444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce1c69-5631-4568-bc8b-a43cde6c1bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:08.729566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce1c69-5631-4568-bc8b-a43cde6c1bff None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:08.729839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce1c69-5631-4568-bc8b-a43cde6c1bff None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:08.730114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce1c69-5631-4568-bc8b-a43cde6c1bff None admin] PATH_INFO: `/v3/users/186838d97c1140bb9bd1a7296d1414bf` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:08.731700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce1c69-5631-4568-bc8b-a43cde6c1bff None admin] RBAC: Authorizing `identity:get_user(user_id=186838d97c1140bb9bd1a7296d1414bf)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:08.733185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce1c69-5631-4568-bc8b-a43cde6c1bff None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:08.734776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 9/16] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:08 2022] GET /identity/v3/users/186838d97c1140bb9bd1a7296d1414bf => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:08.740916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce1c69-5631-4568-bc8b-a43cde6c1bff None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:08.747255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:08.770164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 8/17] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:04:08.775952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c854a14-97bb-4c9e-a88b-4a0af96eba8f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:08.780335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:08.849701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce1c69-5631-4568-bc8b-a43cde6c1bff None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:08.886497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f11a-87d3-4714-9d88-d29449e3e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:08.887177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f11a-87d3-4714-9d88-d29449e3e56e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:08.887484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f11a-87d3-4714-9d88-d29449e3e56e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:08.887755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f11a-87d3-4714-9d88-d29449e3e56e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:08.888344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c854a14-97bb-4c9e-a88b-4a0af96eba8f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:08.891733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f11a-87d3-4714-9d88-d29449e3e56e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:08.891733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f11a-87d3-4714-9d88-d29449e3e56e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:08.893306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:08.898418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f11a-87d3-4714-9d88-d29449e3e56e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:08.936651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:08.937291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:08.937555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:08.937953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:08.949150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:08.950569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:04:08.967523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:08.968508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:04:08.969305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:04:08.969305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:04:08.969305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:08.969305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:04:08.975664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:09.019382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:09.026376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f11a-87d3-4714-9d88-d29449e3e56e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:09.028814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:09.064941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926444d-21be-49c1-a0af-510d45a70121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:09.065695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926444d-21be-49c1-a0af-510d45a70121 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:09.066084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926444d-21be-49c1-a0af-510d45a70121 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:09.066404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926444d-21be-49c1-a0af-510d45a70121 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:09.067370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926444d-21be-49c1-a0af-510d45a70121 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.domain.id=default, user.id=186838d97c1140bb9bd1a7296d1414bf) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:09.067674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926444d-21be-49c1-a0af-510d45a70121 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:09.069123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926444d-21be-49c1-a0af-510d45a70121 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:09.077963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Aug 31 12:04:08 2022] GET /identity/v3/role_assignments?user.id=186838d97c1140bb9bd1a7296d1414bf&scope.domain.id=default&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 233 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:09.099618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a7625-8a6c-44e3-8f90-5ab5b2cea8e6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:09.139514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca148d01-e08d-4677-b2b4-e55fe08fd415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:09.140171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148d01-e08d-4677-b2b4-e55fe08fd415 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:09.140419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148d01-e08d-4677-b2b4-e55fe08fd415 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:09.140692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148d01-e08d-4677-b2b4-e55fe08fd415 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:09.141346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca148d01-e08d-4677-b2b4-e55fe08fd415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:09.141644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca148d01-e08d-4677-b2b4-e55fe08fd415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:09.143080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca148d01-e08d-4677-b2b4-e55fe08fd415 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:09.162699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:11.203607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50da28e-68a0-41f4-8cf0-154a0c2a79df None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:11.204557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50da28e-68a0-41f4-8cf0-154a0c2a79df None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:11.205057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50da28e-68a0-41f4-8cf0-154a0c2a79df None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:11.207456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:11.226808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8d728-9915-4f1e-9673-6776cdc021df None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:11.227373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8d728-9915-4f1e-9673-6776cdc021df None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:11.227373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8d728-9915-4f1e-9673-6776cdc021df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:11.253286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-08a8d728-9915-4f1e-9673-6776cdc021df None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:11.254599 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-08a8d728-9915-4f1e-9673-6776cdc021df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:11.350077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:11.461985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1144b6-9d65-42e3-8338-0dc6cd96722e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:11.462319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1144b6-9d65-42e3-8338-0dc6cd96722e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:11.462509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1144b6-9d65-42e3-8338-0dc6cd96722e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:11.464291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:11.474442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc7829-f214-4f45-8477-cdd6ce91bc56 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:11.474746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc7829-f214-4f45-8477-cdd6ce91bc56 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:11.474991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc7829-f214-4f45-8477-cdd6ce91bc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:11.504863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a6dc7829-f214-4f45-8477-cdd6ce91bc56 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:11.506102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a6dc7829-f214-4f45-8477-cdd6ce91bc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:11.533906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:11.613952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 12/25] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:11.819328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1144b6-9d65-42e3-8338-0dc6cd96722e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:11.961352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1144b6-9d65-42e3-8338-0dc6cd96722e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:11.975345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fde54a-2925-4ca7-941f-43b27d8891e5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:11.975665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fde54a-2925-4ca7-941f-43b27d8891e5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:11.975906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fde54a-2925-4ca7-941f-43b27d8891e5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:11.977811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 13/26] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:11.985047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfc73a-2db4-4559-a121-ce8af68d7e88 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:11.985350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfc73a-2db4-4559-a121-ce8af68d7e88 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:11.985609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfc73a-2db4-4559-a121-ce8af68d7e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.028809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4acfc73a-2db4-4559-a121-ce8af68d7e88 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:12.030628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4acfc73a-2db4-4559-a121-ce8af68d7e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:12.043948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fdea3-c0c4-4a6b-969f-981ea27873ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:12.045129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fdea3-c0c4-4a6b-969f-981ea27873ed None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:12.045601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fdea3-c0c4-4a6b-969f-981ea27873ed None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:12.046124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fdea3-c0c4-4a6b-969f-981ea27873ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.079279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fdea3-c0c4-4a6b-969f-981ea27873ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5b3fdea3-c0c4-4a6b-969f-981ea27873ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:12.099492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:12.100620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:12.101689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:12.102983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 14/27] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:12.105023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fdea3-c0c4-4a6b-969f-981ea27873ed None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:12.158226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 14/28] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:12.159318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfc73a-2db4-4559-a121-ce8af68d7e88 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:12.167314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:12.297589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fdea3-c0c4-4a6b-969f-981ea27873ed None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:12.366047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:12.374261 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2458a-1359-4e47-86c9-23b82021ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:12.375244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2458a-1359-4e47-86c9-23b82021ca51 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:12.375719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2458a-1359-4e47-86c9-23b82021ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:12.376198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2458a-1359-4e47-86c9-23b82021ca51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.395200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfc73a-2db4-4559-a121-ce8af68d7e88 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:12.412192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2458a-1359-4e47-86c9-23b82021ca51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:12.415659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2458a-1359-4e47-86c9-23b82021ca51 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8fa2458a-1359-4e47-86c9-23b82021ca51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:04:12.437687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:12.438595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:04:12.439498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:04:12.439498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:04:12.439498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:12.441441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 15/29] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:12.442945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ec5ce-61ef-4ea9-b09c-847af642d374 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:12.443605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ec5ce-61ef-4ea9-b09c-847af642d374 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:12.444099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ec5ce-61ef-4ea9-b09c-847af642d374 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.445809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 16/30] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:12.452316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ec5ce-61ef-4ea9-b09c-847af642d374 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:12.481369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d38fc-a6de-4867-9776-bb9d4f7239c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:12.483555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d38fc-a6de-4867-9776-bb9d4f7239c7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:12.484569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d38fc-a6de-4867-9776-bb9d4f7239c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:12.487371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d38fc-a6de-4867-9776-bb9d4f7239c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.488653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d38fc-a6de-4867-9776-bb9d4f7239c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:12.489124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d38fc-a6de-4867-9776-bb9d4f7239c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:12.490923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d38fc-a6de-4867-9776-bb9d4f7239c7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:12.521381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 15/31] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:12.525001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d761bf-3600-46cd-a484-74c33708753d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:12.525499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d761bf-3600-46cd-a484-74c33708753d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:12.525786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d761bf-3600-46cd-a484-74c33708753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.560411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-27d761bf-3600-46cd-a484-74c33708753d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:12.563339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-27d761bf-3600-46cd-a484-74c33708753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:12.602038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:12.639614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ec5ce-61ef-4ea9-b09c-847af642d374 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:12.645922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 16/32] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:12.646774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d761bf-3600-46cd-a484-74c33708753d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:12.685334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:12.685958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:12.686234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:12.686503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.687148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:12.687470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:12.688875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:12.725202 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 17/33] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:12.727680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:12.776829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:12.829681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d761bf-3600-46cd-a484-74c33708753d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:12.869186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bc95d-32b8-497e-8337-becaa1f6bf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:12.870325 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bc95d-32b8-497e-8337-becaa1f6bf01 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:12.875940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bc95d-32b8-497e-8337-becaa1f6bf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:12.876531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bc95d-32b8-497e-8337-becaa1f6bf01 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.877461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bc95d-32b8-497e-8337-becaa1f6bf01 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:12.879412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bc95d-32b8-497e-8337-becaa1f6bf01 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:12.930802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef52ff-04be-4eb9-aac3-62cf6e0202e4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:12.985786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc3b7b-9ff4-405e-b3f2-d401e55a3f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:12.986422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc3b7b-9ff4-405e-b3f2-d401e55a3f4b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:12.986696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc3b7b-9ff4-405e-b3f2-d401e55a3f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:12.986965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc3b7b-9ff4-405e-b3f2-d401e55a3f4b None admin] PATH_INFO: `/v3/users/186838d97c1140bb9bd1a7296d1414bf` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:12.990268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc3b7b-9ff4-405e-b3f2-d401e55a3f4b None admin] RBAC: Authorizing `identity:get_user(user_id=186838d97c1140bb9bd1a7296d1414bf)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:12.991814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc3b7b-9ff4-405e-b3f2-d401e55a3f4b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:12.993412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 18/34] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:12 2022] GET /identity/v3/users/186838d97c1140bb9bd1a7296d1414bf => generated 247 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:12.995187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc3b7b-9ff4-405e-b3f2-d401e55a3f4b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:13.006797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:13.040336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 17/35] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 31 12:04:12 2022] POST /identity/v3/roles => generated 225 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:13.041393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bc95d-32b8-497e-8337-becaa1f6bf01 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:13.113567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc3b7b-9ff4-405e-b3f2-d401e55a3f4b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:13.152166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcc9b1-8ecd-4188-9e76-a53d532a418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:13.152732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcc9b1-8ecd-4188-9e76-a53d532a418a None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:13.153009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcc9b1-8ecd-4188-9e76-a53d532a418a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:13.153276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcc9b1-8ecd-4188-9e76-a53d532a418a None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:13.153958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcc9b1-8ecd-4188-9e76-a53d532a418a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:13.155625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcc9b1-8ecd-4188-9e76-a53d532a418a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:13.196461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bc95d-32b8-497e-8337-becaa1f6bf01 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:13.241857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692ed7a-e369-4f32-8ff1-af94e75f789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:13.242628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692ed7a-e369-4f32-8ff1-af94e75f789b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:13.242911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692ed7a-e369-4f32-8ff1-af94e75f789b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:13.243182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692ed7a-e369-4f32-8ff1-af94e75f789b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:13.245160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692ed7a-e369-4f32-8ff1-af94e75f789b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:13.246866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692ed7a-e369-4f32-8ff1-af94e75f789b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:13.248886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 18/36] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:13.249634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692ed7a-e369-4f32-8ff1-af94e75f789b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:13.270897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:13.320154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 19/37] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:04:12 2022] POST /identity/v3/projects => generated 301 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:13.321486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065103a8-66f0-4f10-9a05-9a2284ebd316 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:13.321778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065103a8-66f0-4f10-9a05-9a2284ebd316 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:13.322055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065103a8-66f0-4f10-9a05-9a2284ebd316 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:13.323493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 20/38] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:13.325187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065103a8-66f0-4f10-9a05-9a2284ebd316 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:13.393486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692ed7a-e369-4f32-8ff1-af94e75f789b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:13.404603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:13.436223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b50cab-713b-4be1-973f-eaac20b7b564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:13.437002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50cab-713b-4be1-973f-eaac20b7b564 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:13.437287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50cab-713b-4be1-973f-eaac20b7b564 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:13.437562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50cab-713b-4be1-973f-eaac20b7b564 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:13.438256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b50cab-713b-4be1-973f-eaac20b7b564 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:13.439864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b50cab-713b-4be1-973f-eaac20b7b564 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:13.455562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065103a8-66f0-4f10-9a05-9a2284ebd316 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:13.463290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 19/39] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 31 12:04:13 2022] POST /identity/v3/roles => generated 219 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:13.463290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285502e-ac91-4529-9968-4c55e0aed831 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:13.463290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285502e-ac91-4529-9968-4c55e0aed831 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:13.463290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285502e-ac91-4529-9968-4c55e0aed831 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:13.463290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 20/40] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:13.463290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac20d28-8df4-4314-9ab5-bfb6ae005fd5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:13.463786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac20d28-8df4-4314-9ab5-bfb6ae005fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:13.463969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac20d28-8df4-4314-9ab5-bfb6ae005fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:13.509379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8fe55-3e96-43ab-a196-dd9f5a0b19ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:13.510159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8fe55-3e96-43ab-a196-dd9f5a0b19ea None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:13.510448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8fe55-3e96-43ab-a196-dd9f5a0b19ea None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:13.510703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8fe55-3e96-43ab-a196-dd9f5a0b19ea None admin] PATH_INFO: `/v3/domains/default/users/186838d97c1140bb9bd1a7296d1414bf/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:13.520450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8fe55-3e96-43ab-a196-dd9f5a0b19ea None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=186838d97c1140bb9bd1a7296d1414bf, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:13.522294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8fe55-3e96-43ab-a196-dd9f5a0b19ea None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:13.541641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 21/41] 10.222.0.27 () {58 vars in 1361 bytes} [Wed Aug 31 12:04:13 2022] PUT /identity/v3/domains/default/users/186838d97c1140bb9bd1a7296d1414bf/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:13.543474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852511-fde4-4e34-aea7-c5d29aa28341 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:13.543763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852511-fde4-4e34-aea7-c5d29aa28341 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:13.544049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852511-fde4-4e34-aea7-c5d29aa28341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:13.567843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b3852511-fde4-4e34-aea7-c5d29aa28341 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:13.569869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b3852511-fde4-4e34-aea7-c5d29aa28341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:13.604559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1ac20d28-8df4-4314-9ab5-bfb6ae005fd5 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:13.606185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1ac20d28-8df4-4314-9ab5-bfb6ae005fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:13.673249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 22/42] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:13.709917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:13.718952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 21/43] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:13.940231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3852511-fde4-4e34-aea7-c5d29aa28341 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:13.984636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac20d28-8df4-4314-9ab5-bfb6ae005fd5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:13.994912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:14.055953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3852511-fde4-4e34-aea7-c5d29aa28341 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:14.094671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac20d28-8df4-4314-9ab5-bfb6ae005fd5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:14.094878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be544f-6b12-4284-9023-295f6eefa637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:14.095352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be544f-6b12-4284-9023-295f6eefa637 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:14.095639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be544f-6b12-4284-9023-295f6eefa637 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:14.095912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be544f-6b12-4284-9023-295f6eefa637 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:14.096744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be544f-6b12-4284-9023-295f6eefa637 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:14.098059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be544f-6b12-4284-9023-295f6eefa637 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:14.115973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 23/44] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 31 12:04:13 2022] POST /identity/v3/roles => generated 223 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:14.137347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792a4c2-7459-4026-9def-74af8d6c5845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:14.138185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792a4c2-7459-4026-9def-74af8d6c5845 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:14.138563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792a4c2-7459-4026-9def-74af8d6c5845 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:14.138934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792a4c2-7459-4026-9def-74af8d6c5845 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:14.140329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792a4c2-7459-4026-9def-74af8d6c5845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:14.141967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792a4c2-7459-4026-9def-74af8d6c5845 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:14.144818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 22/45] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:14.150912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be544f-6b12-4284-9023-295f6eefa637 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:14.266303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be544f-6b12-4284-9023-295f6eefa637 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:14.276433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:14.305667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff1278-d80a-4091-9ed5-8fdaa9a3c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:14.306302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1278-d80a-4091-9ed5-8fdaa9a3c5e4 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:14.306577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1278-d80a-4091-9ed5-8fdaa9a3c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:14.306855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1278-d80a-4091-9ed5-8fdaa9a3c5e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:14.307612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff1278-d80a-4091-9ed5-8fdaa9a3c5e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:14.309094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff1278-d80a-4091-9ed5-8fdaa9a3c5e4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:14.337286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:04:14 2022] POST /identity/v3/projects => generated 312 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:04:15.514833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48478781-74f3-4916-8067-ff598ae0b6a2 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:15.515399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48478781-74f3-4916-8067-ff598ae0b6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:15.515399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48478781-74f3-4916-8067-ff598ae0b6a2 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:15.516736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 23/47] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:15.522317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff10c8-6969-4326-a361-b82a29879367 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:15.522718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff10c8-6969-4326-a361-b82a29879367 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:15.522898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff10c8-6969-4326-a361-b82a29879367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:15.541830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-02ff10c8-6969-4326-a361-b82a29879367 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:15.543011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-02ff10c8-6969-4326-a361-b82a29879367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:15.615420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 25/48] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:15.881265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48478781-74f3-4916-8067-ff598ae0b6a2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:15.949214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:15.992356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48478781-74f3-4916-8067-ff598ae0b6a2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:16.028207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fdc6c-a71f-471c-80a0-67bd34b5f204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:16.028845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fdc6c-a71f-471c-80a0-67bd34b5f204 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.029125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fdc6c-a71f-471c-80a0-67bd34b5f204 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.029371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fdc6c-a71f-471c-80a0-67bd34b5f204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.040176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fdc6c-a71f-471c-80a0-67bd34b5f204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c51fdc6c-a71f-471c-80a0-67bd34b5f204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:16.045068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:16.045930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:16.046727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 24/49] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:16.052633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff10c8-6969-4326-a361-b82a29879367 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:16.147699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:16.159727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff10c8-6969-4326-a361-b82a29879367 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:16.196308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a634c2-2e35-4a9a-ace3-9b1088dd23af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:16.196929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a634c2-2e35-4a9a-ace3-9b1088dd23af None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.197212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a634c2-2e35-4a9a-ace3-9b1088dd23af None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.197492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a634c2-2e35-4a9a-ace3-9b1088dd23af None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.198161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a634c2-2e35-4a9a-ace3-9b1088dd23af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:16.198470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a634c2-2e35-4a9a-ace3-9b1088dd23af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:16.199930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a634c2-2e35-4a9a-ace3-9b1088dd23af None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:16.208285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 26/50] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:16.214480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fdc6c-a71f-471c-80a0-67bd34b5f204 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:16.221099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c4785-b4ad-4aa5-ae1d-9f0866ef23b8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.221390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c4785-b4ad-4aa5-ae1d-9f0866ef23b8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.221663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c4785-b4ad-4aa5-ae1d-9f0866ef23b8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.223200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 27/51] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:04:16.228474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f081915-def8-45a5-8dbd-85f4bb69484b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.228767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f081915-def8-45a5-8dbd-85f4bb69484b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.229046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f081915-def8-45a5-8dbd-85f4bb69484b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.247456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0f081915-def8-45a5-8dbd-85f4bb69484b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:16.248672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0f081915-def8-45a5-8dbd-85f4bb69484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:16.315113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fdc6c-a71f-471c-80a0-67bd34b5f204 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:16.319935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 28/52] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:16.339136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:16.352217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe29ee-d056-45c6-8801-0b57c968093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:16.352856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe29ee-d056-45c6-8801-0b57c968093e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.353131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe29ee-d056-45c6-8801-0b57c968093e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.353384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe29ee-d056-45c6-8801-0b57c968093e None admin] PATH_INFO: `/v3/users/186838d97c1140bb9bd1a7296d1414bf` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.354515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe29ee-d056-45c6-8801-0b57c968093e None admin] RBAC: Authorizing `identity:get_user(user_id=186838d97c1140bb9bd1a7296d1414bf)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:16.356030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe29ee-d056-45c6-8801-0b57c968093e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:16.362664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 25/53] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:16 2022] GET /identity/v3/users/186838d97c1140bb9bd1a7296d1414bf => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:16.364103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f081915-def8-45a5-8dbd-85f4bb69484b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:16.553819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:16.571768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f081915-def8-45a5-8dbd-85f4bb69484b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:16.582975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe29ee-d056-45c6-8801-0b57c968093e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:16.605722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329dab9-409b-4cf1-90ef-dcc52b61b0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:16.606362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329dab9-409b-4cf1-90ef-dcc52b61b0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.606637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329dab9-409b-4cf1-90ef-dcc52b61b0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.606900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329dab9-409b-4cf1-90ef-dcc52b61b0c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.608034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329dab9-409b-4cf1-90ef-dcc52b61b0c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:16.609481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329dab9-409b-4cf1-90ef-dcc52b61b0c8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:16.611064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 29/54] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:16.617159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329dab9-409b-4cf1-90ef-dcc52b61b0c8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:16.690785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe29ee-d056-45c6-8801-0b57c968093e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:16.716291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329dab9-409b-4cf1-90ef-dcc52b61b0c8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:16.725056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb8592-2285-43ba-9766-d2989a862f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:16.725690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb8592-2285-43ba-9766-d2989a862f5c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.726298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb8592-2285-43ba-9766-d2989a862f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.726569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb8592-2285-43ba-9766-d2989a862f5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.727709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb8592-2285-43ba-9766-d2989a862f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:16.729172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb8592-2285-43ba-9766-d2989a862f5c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:16.730758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 26/55] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:16.736628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb8592-2285-43ba-9766-d2989a862f5c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:16.750973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c115b4-cf70-4213-b521-c54913070d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:16.751628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c115b4-cf70-4213-b521-c54913070d58 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.751912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c115b4-cf70-4213-b521-c54913070d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.752185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c115b4-cf70-4213-b521-c54913070d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.753086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c115b4-cf70-4213-b521-c54913070d58 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.domain.id=default, user.id=186838d97c1140bb9bd1a7296d1414bf) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:16.753411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c115b4-cf70-4213-b521-c54913070d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:16.754896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c115b4-cf70-4213-b521-c54913070d58 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:16.760715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 30/56] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Aug 31 12:04:16 2022] GET /identity/v3/role_assignments?user.id=186838d97c1140bb9bd1a7296d1414bf&scope.domain.id=default&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:16.806369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:16.855253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb8592-2285-43ba-9766-d2989a862f5c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:16.891943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3929-4afb-4fcf-a40c-99e7f8176b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:16.892580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3929-4afb-4fcf-a40c-99e7f8176b89 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:16.892859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3929-4afb-4fcf-a40c-99e7f8176b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:16.893300 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3929-4afb-4fcf-a40c-99e7f8176b89 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:16.893902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a3929-4afb-4fcf-a40c-99e7f8176b89 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:16.895448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a3929-4afb-4fcf-a40c-99e7f8176b89 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:16.927627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 27/57] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Aug 31 12:04:16 2022] POST /identity/v3/projects => generated 298 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:04:18.847112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0714-c48a-4749-857e-6595173fff52 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:18.847730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0714-c48a-4749-857e-6595173fff52 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:18.847817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0714-c48a-4749-857e-6595173fff52 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:18.849880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 31/58] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:18.855550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbca24d-e219-456f-b0f3-14a3fb56a35b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:18.855984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbca24d-e219-456f-b0f3-14a3fb56a35b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:18.856088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbca24d-e219-456f-b0f3-14a3fb56a35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:18.875871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5bbca24d-e219-456f-b0f3-14a3fb56a35b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:18.877084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5bbca24d-e219-456f-b0f3-14a3fb56a35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:18.951343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 28/59] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:19.215748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca0714-c48a-4749-857e-6595173fff52 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:19.233815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:19.327767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca0714-c48a-4749-857e-6595173fff52 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:19.363208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbc8b3-0ab2-4456-833a-6e82108240c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:19.363884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbc8b3-0ab2-4456-833a-6e82108240c9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:19.364170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbc8b3-0ab2-4456-833a-6e82108240c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:19.364438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbc8b3-0ab2-4456-833a-6e82108240c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:19.365634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbc8b3-0ab2-4456-833a-6e82108240c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:19.367098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbc8b3-0ab2-4456-833a-6e82108240c9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:19.368698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 32/60] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:19.374576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbca24d-e219-456f-b0f3-14a3fb56a35b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:19.418801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:19.482134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbca24d-e219-456f-b0f3-14a3fb56a35b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:19.518016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c3ac1-10d4-41a4-891f-b7fef831457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:19.518632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3ac1-10d4-41a4-891f-b7fef831457e None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:19.518913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3ac1-10d4-41a4-891f-b7fef831457e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:19.519185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3ac1-10d4-41a4-891f-b7fef831457e None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:19.519883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c3ac1-10d4-41a4-891f-b7fef831457e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:19.521387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c3ac1-10d4-41a4-891f-b7fef831457e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:19.549836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 29/61] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:04:19 2022] POST /identity/v3/users => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:22.517686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1dd70-1dbb-49b3-a287-24ce5981c7c9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:22.518698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1dd70-1dbb-49b3-a287-24ce5981c7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:22.519169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1dd70-1dbb-49b3-a287-24ce5981c7c9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:22.520738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 33/62] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:22.526399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e587-f148-4954-b973-82f4e2381446 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:22.526849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e587-f148-4954-b973-82f4e2381446 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:22.527025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e587-f148-4954-b973-82f4e2381446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:22.556352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cdb5e587-f148-4954-b973-82f4e2381446 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:22.557629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cdb5e587-f148-4954-b973-82f4e2381446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:22.652933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 30/63] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:22.741914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59ce2f-7fc8-416f-8c2e-8b8e0003922d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:22.742534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59ce2f-7fc8-416f-8c2e-8b8e0003922d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:22.742998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59ce2f-7fc8-416f-8c2e-8b8e0003922d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:22.744556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 34/64] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:22.749968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2fdae-7ca8-4c07-be0b-31ea74a278bc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:22.750319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2fdae-7ca8-4c07-be0b-31ea74a278bc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:22.750603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2fdae-7ca8-4c07-be0b-31ea74a278bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:22.776369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-73c2fdae-7ca8-4c07-be0b-31ea74a278bc None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:22.777563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-73c2fdae-7ca8-4c07-be0b-31ea74a278bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:22.808221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:22.877387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 31/65] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:22.961866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459310-55b6-4f0f-9d83-be4648749dde None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:22.962537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459310-55b6-4f0f-9d83-be4648749dde None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:22.963092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459310-55b6-4f0f-9d83-be4648749dde None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:22.964711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 35/66] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:22.973617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290dc33-5905-4719-b3c0-6cbca09f250b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:22.973908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290dc33-5905-4719-b3c0-6cbca09f250b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:22.974192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290dc33-5905-4719-b3c0-6cbca09f250b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.004297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a290dc33-5905-4719-b3c0-6cbca09f250b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:23.005857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a290dc33-5905-4719-b3c0-6cbca09f250b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:23.093661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 32/67] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:23.098217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78459310-55b6-4f0f-9d83-be4648749dde None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:23.135503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954d7af-4afc-4396-b904-e5297f941bc8 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.136116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954d7af-4afc-4396-b904-e5297f941bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.136693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954d7af-4afc-4396-b904-e5297f941bc8 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.138463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 33/68] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:23.143666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739c4f4-d44e-4ccd-a0c5-66c4e0ac8c5c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.144198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739c4f4-d44e-4ccd-a0c5-66c4e0ac8c5c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.144677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739c4f4-d44e-4ccd-a0c5-66c4e0ac8c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.173416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1739c4f4-d44e-4ccd-a0c5-66c4e0ac8c5c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:23.175691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1739c4f4-d44e-4ccd-a0c5-66c4e0ac8c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:23.262426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 34/69] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:23.271255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739c4f4-d44e-4ccd-a0c5-66c4e0ac8c5c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:23.275266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:23.296744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:23.318998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78459310-55b6-4f0f-9d83-be4648749dde None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:23.394225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8306fa4-bda9-4d3d-94d8-f1d2149824e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:23.395756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8306fa4-bda9-4d3d-94d8-f1d2149824e5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.396254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8306fa4-bda9-4d3d-94d8-f1d2149824e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.396706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8306fa4-bda9-4d3d-94d8-f1d2149824e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.408744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8306fa4-bda9-4d3d-94d8-f1d2149824e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f8306fa4-bda9-4d3d-94d8-f1d2149824e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:23.415082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:23.415959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:23.416710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:23.418939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 36/70] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:04:23.425483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8306fa4-bda9-4d3d-94d8-f1d2149824e5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:23.472649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739c4f4-d44e-4ccd-a0c5-66c4e0ac8c5c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:23.510231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6236e85-a7b0-48ad-8a33-2459083e159b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:23.510884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236e85-a7b0-48ad-8a33-2459083e159b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.511152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236e85-a7b0-48ad-8a33-2459083e159b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.511455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236e85-a7b0-48ad-8a33-2459083e159b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.522077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6236e85-a7b0-48ad-8a33-2459083e159b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a6236e85-a7b0-48ad-8a33-2459083e159b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:23.526901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:23.528292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:23.529234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 35/71] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:23.534626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efaf9b-df4b-4b48-8987-de39ea31bfa4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.534911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efaf9b-df4b-4b48-8987-de39ea31bfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.535187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efaf9b-df4b-4b48-8987-de39ea31bfa4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.536661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 36/72] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:23.537222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efaf9b-df4b-4b48-8987-de39ea31bfa4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:23.540029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8306fa4-bda9-4d3d-94d8-f1d2149824e5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:23.594787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:23.595503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.595852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.596177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.596925 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:23.597239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:23.598725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:23.608387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 37/73] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:23.609167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:23.613474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:23.640193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:23.667684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efaf9b-df4b-4b48-8987-de39ea31bfa4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:23.704777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41192d95-e595-4129-8e9c-f8dcdd2b470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:23.706129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192d95-e595-4129-8e9c-f8dcdd2b470f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.706412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192d95-e595-4129-8e9c-f8dcdd2b470f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.706686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192d95-e595-4129-8e9c-f8dcdd2b470f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.716696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41192d95-e595-4129-8e9c-f8dcdd2b470f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-41192d95-e595-4129-8e9c-f8dcdd2b470f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:23.722015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:23.723075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:23.724087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 37/74] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:23.725002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a70f1-ff1a-4255-9e9b-17084458444f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.725002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a70f1-ff1a-4255-9e9b-17084458444f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.725002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a70f1-ff1a-4255-9e9b-17084458444f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.732068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb66e66-dcb6-4bae-aa4f-8a00ebe2e5e7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:23.749919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-990a70f1-ff1a-4255-9e9b-17084458444f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:23.751104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-990a70f1-ff1a-4255-9e9b-17084458444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:23.773206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:23.773834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.774121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.774390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.775061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:23.775407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:23.776786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:23.785757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 38/75] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:23.786353 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:23.826218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 38/76] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:23.827169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a70f1-ff1a-4255-9e9b-17084458444f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:23.856697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:23.898206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:23.904805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30c3b3-1873-4d0f-b49a-223494630359 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:23.935753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a70f1-ff1a-4255-9e9b-17084458444f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:23.941709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d2125-4425-4be1-9bb9-00bc4c91d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:23.942345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d2125-4425-4be1-9bb9-00bc4c91d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.942625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d2125-4425-4be1-9bb9-00bc4c91d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.943006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d2125-4425-4be1-9bb9-00bc4c91d1a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.954890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d2125-4425-4be1-9bb9-00bc4c91d1a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-074d2125-4425-4be1-9bb9-00bc4c91d1a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:23.961564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:23.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:23.963256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:23.963860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 39/77] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:23.964489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d2125-4425-4be1-9bb9-00bc4c91d1a5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:23.979046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7329a-aeaa-4335-88e5-7092fead5e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:23.980015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7329a-aeaa-4335-88e5-7092fead5e32 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:23.980430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7329a-aeaa-4335-88e5-7092fead5e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:23.980835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7329a-aeaa-4335-88e5-7092fead5e32 None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:23.996505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7329a-aeaa-4335-88e5-7092fead5e32 None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:23.998000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7329a-aeaa-4335-88e5-7092fead5e32 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:23.999884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 39/78] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:24.000448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7329a-aeaa-4335-88e5-7092fead5e32 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.073233 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d2125-4425-4be1-9bb9-00bc4c91d1a5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.100471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:24.110361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7329a-aeaa-4335-88e5-7092fead5e32 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.111104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.111823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.112099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.112369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.113038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:24.113341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.114725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.122779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 40/79] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:24.123680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.139540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:24.151077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6117c-40a2-4ff4-b24b-d589aebc11ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.151754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6117c-40a2-4ff4-b24b-d589aebc11ee None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.152035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6117c-40a2-4ff4-b24b-d589aebc11ee None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.152313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6117c-40a2-4ff4-b24b-d589aebc11ee None admin] PATH_INFO: `/v3/users/186838d97c1140bb9bd1a7296d1414bf` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.153446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6117c-40a2-4ff4-b24b-d589aebc11ee None admin] RBAC: Authorizing `identity:get_user(user_id=186838d97c1140bb9bd1a7296d1414bf)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.154898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6117c-40a2-4ff4-b24b-d589aebc11ee None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.156527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 40/80] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:23 2022] GET /identity/v3/users/186838d97c1140bb9bd1a7296d1414bf => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:24.161477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6117c-40a2-4ff4-b24b-d589aebc11ee None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.225313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ba1a1-b366-4407-b32d-8d40067380be None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.259881 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.260489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.260767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.261037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.261692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:24.262039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6117c-40a2-4ff4-b24b-d589aebc11ee None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.262135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.263454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.271871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 41/81] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:24.272659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.296726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621c577-347e-4171-bba0-bf4d50ef9310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.297334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621c577-347e-4171-bba0-bf4d50ef9310 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.297609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621c577-347e-4171-bba0-bf4d50ef9310 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.297884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621c577-347e-4171-bba0-bf4d50ef9310 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.322951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:24.358519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:24.369872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621c577-347e-4171-bba0-bf4d50ef9310 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.370964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cca775-2786-4ab8-b475-44a71c7b88d6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.371382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621c577-347e-4171-bba0-bf4d50ef9310 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.404563 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfcad3-9a97-4661-8ade-536ee605fc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.405186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfcad3-9a97-4661-8ade-536ee605fc18 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.405461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfcad3-9a97-4661-8ade-536ee605fc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.405730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfcad3-9a97-4661-8ade-536ee605fc18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.406856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfcad3-9a97-4661-8ade-536ee605fc18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.408825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfcad3-9a97-4661-8ade-536ee605fc18 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.410329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 42/82] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:24.410851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfcad3-9a97-4661-8ade-536ee605fc18 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.468664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 41/83] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:24.469514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621c577-347e-4171-bba0-bf4d50ef9310 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.510225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfcad3-9a97-4661-8ade-536ee605fc18 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.528385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:24.545618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235c81f-f45a-42c4-8d6b-ede52ffa693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.546264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235c81f-f45a-42c4-8d6b-ede52ffa693a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.546543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235c81f-f45a-42c4-8d6b-ede52ffa693a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.546818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235c81f-f45a-42c4-8d6b-ede52ffa693a None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.547992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235c81f-f45a-42c4-8d6b-ede52ffa693a None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.549961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235c81f-f45a-42c4-8d6b-ede52ffa693a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.551276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 43/84] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:24.552051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235c81f-f45a-42c4-8d6b-ede52ffa693a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.559439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:24.568606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621c577-347e-4171-bba0-bf4d50ef9310 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.602167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99f1fd-f971-4b12-89da-8e5aef6cb2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.602778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f1fd-f971-4b12-89da-8e5aef6cb2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.603059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f1fd-f971-4b12-89da-8e5aef6cb2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.603364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f1fd-f971-4b12-89da-8e5aef6cb2e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.653939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235c81f-f45a-42c4-8d6b-ede52ffa693a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.671608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99f1fd-f971-4b12-89da-8e5aef6cb2e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.673115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99f1fd-f971-4b12-89da-8e5aef6cb2e7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.687641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244268a5-7a81-4b4e-99aa-eeb765135b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.688272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244268a5-7a81-4b4e-99aa-eeb765135b80 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.688539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244268a5-7a81-4b4e-99aa-eeb765135b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.688811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244268a5-7a81-4b4e-99aa-eeb765135b80 None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.689985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244268a5-7a81-4b4e-99aa-eeb765135b80 None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.691489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244268a5-7a81-4b4e-99aa-eeb765135b80 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.693078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 44/85] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:24.693910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244268a5-7a81-4b4e-99aa-eeb765135b80 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.749039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:24.765005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:24.774776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 42/86] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:24.775635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99f1fd-f971-4b12-89da-8e5aef6cb2e7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.794109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244268a5-7a81-4b4e-99aa-eeb765135b80 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.827788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e057a-d677-46f9-8431-dfd47a8665c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.828399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e057a-d677-46f9-8431-dfd47a8665c1 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.828703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e057a-d677-46f9-8431-dfd47a8665c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.828982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e057a-d677-46f9-8431-dfd47a8665c1 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.829638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e057a-d677-46f9-8431-dfd47a8665c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.831093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e057a-d677-46f9-8431-dfd47a8665c1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.859036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 45/87] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:04:24 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:24.864270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e057a-d677-46f9-8431-dfd47a8665c1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.880500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99f1fd-f971-4b12-89da-8e5aef6cb2e7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:24.922386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632be7c0-ac7f-4bf5-aa7c-9c149aafcb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:24.923044 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be7c0-ac7f-4bf5-aa7c-9c149aafcb44 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:24.923359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be7c0-ac7f-4bf5-aa7c-9c149aafcb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:24.923636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be7c0-ac7f-4bf5-aa7c-9c149aafcb44 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:24.925048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632be7c0-ac7f-4bf5-aa7c-9c149aafcb44 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:24.926550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632be7c0-ac7f-4bf5-aa7c-9c149aafcb44 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:24.928337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 43/88] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:24.929153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632be7c0-ac7f-4bf5-aa7c-9c149aafcb44 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:24.992839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:25.000101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e057a-d677-46f9-8431-dfd47a8665c1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.013654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:25.046253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632be7c0-ac7f-4bf5-aa7c-9c149aafcb44 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.052460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d9d0a-ddeb-425d-9869-04dd51f7ce04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.053132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d9d0a-ddeb-425d-9869-04dd51f7ce04 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.053410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d9d0a-ddeb-425d-9869-04dd51f7ce04 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.053686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d9d0a-ddeb-425d-9869-04dd51f7ce04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.088299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b63490-2dd8-4f68-8416-4cc8ea2e1863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.088930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b63490-2dd8-4f68-8416-4cc8ea2e1863 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.089215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b63490-2dd8-4f68-8416-4cc8ea2e1863 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.089495 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b63490-2dd8-4f68-8416-4cc8ea2e1863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.141878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d9d0a-ddeb-425d-9869-04dd51f7ce04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.144110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d9d0a-ddeb-425d-9869-04dd51f7ce04 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.173754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b63490-2dd8-4f68-8416-4cc8ea2e1863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.175327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b63490-2dd8-4f68-8416-4cc8ea2e1863 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.244930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:25.259985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 46/89] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:25.260864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d9d0a-ddeb-425d-9869-04dd51f7ce04 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:25.272863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:25.299105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 44/90] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:25.300652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b63490-2dd8-4f68-8416-4cc8ea2e1863 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:25.370219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d9d0a-ddeb-425d-9869-04dd51f7ce04 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.419728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae691c9-a439-4f73-99d5-c671206bee12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.420773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae691c9-a439-4f73-99d5-c671206bee12 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.421349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae691c9-a439-4f73-99d5-c671206bee12 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.421823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae691c9-a439-4f73-99d5-c671206bee12 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.423191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae691c9-a439-4f73-99d5-c671206bee12 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.425462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae691c9-a439-4f73-99d5-c671206bee12 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.427434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 47/91] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:25 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:25.427558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b63490-2dd8-4f68-8416-4cc8ea2e1863 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.428521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae691c9-a439-4f73-99d5-c671206bee12 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:25.473732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.474359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.474674 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.475357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.477662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] RBAC: Adding query filter params (role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=de5700cfe30946bd8e0082d5b4964f98, user.id=16aab776604b46f6b9f706d4d684a6b9) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:25.477979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.479554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.487877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 45/92] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:25 2022] GET /identity/v3/role_assignments?user.id=16aab776604b46f6b9f706d4d684a6b9&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:25.488727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:25.517496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:25.528433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:25.561113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae691c9-a439-4f73-99d5-c671206bee12 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.608390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7b7bd-2d2f-4ae8-8fcf-ccd11ed9d052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.609189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b7bd-2d2f-4ae8-8fcf-ccd11ed9d052 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.609561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b7bd-2d2f-4ae8-8fcf-ccd11ed9d052 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.609933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b7bd-2d2f-4ae8-8fcf-ccd11ed9d052 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.611219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7b7bd-2d2f-4ae8-8fcf-ccd11ed9d052 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.613360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7b7bd-2d2f-4ae8-8fcf-ccd11ed9d052 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.615132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 48/93] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:25 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:25.616072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7b7bd-2d2f-4ae8-8fcf-ccd11ed9d052 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:25.617323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b46d0-44c8-4c2f-8017-8324527e7175 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.675466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ff246-eee3-4acc-acd7-ce72279f8d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.676157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ff246-eee3-4acc-acd7-ce72279f8d38 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.676441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ff246-eee3-4acc-acd7-ce72279f8d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.676718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ff246-eee3-4acc-acd7-ce72279f8d38 None admin] PATH_INFO: `/v3/projects/868e9fb5616543f6b193d6f1b79ffef3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.680540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ff246-eee3-4acc-acd7-ce72279f8d38 None admin] RBAC: Authorizing `identity:get_project(project_id=868e9fb5616543f6b193d6f1b79ffef3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.682053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ff246-eee3-4acc-acd7-ce72279f8d38 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.688656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 46/94] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:25 2022] GET /identity/v3/projects/868e9fb5616543f6b193d6f1b79ffef3 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:25.689830 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ff246-eee3-4acc-acd7-ce72279f8d38 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:25.764290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7b7bd-2d2f-4ae8-8fcf-ccd11ed9d052 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.803879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.804497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.804788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.805069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.805791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=de5700cfe30946bd8e0082d5b4964f98, user.id=186838d97c1140bb9bd1a7296d1414bf) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:25.806107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.808117 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.814500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:25.815871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 49/95] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:25 2022] GET /identity/v3/role_assignments?user.id=186838d97c1140bb9bd1a7296d1414bf&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:25.816805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:25.825192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ff246-eee3-4acc-acd7-ce72279f8d38 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.833306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:25.865114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094128c5-6538-466c-84c8-bf6170cb2249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.866128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094128c5-6538-466c-84c8-bf6170cb2249 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.866601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094128c5-6538-466c-84c8-bf6170cb2249 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.867120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094128c5-6538-466c-84c8-bf6170cb2249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.868065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094128c5-6538-466c-84c8-bf6170cb2249 None admin] RBAC: Adding query filter params (role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=de5700cfe30946bd8e0082d5b4964f98, user.id=16aab776604b46f6b9f706d4d684a6b9) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:25.868585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094128c5-6538-466c-84c8-bf6170cb2249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.870233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094128c5-6538-466c-84c8-bf6170cb2249 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.876745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 47/96] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:25 2022] GET /identity/v3/role_assignments?user.id=16aab776604b46f6b9f706d4d684a6b9&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:25.941285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df344aa-e1c7-499e-90bd-04a0845a8421 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:25.980242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaadbd-5cb2-41ce-b10d-bc35a2062895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:25.980876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaadbd-5cb2-41ce-b10d-bc35a2062895 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:25.981156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaadbd-5cb2-41ce-b10d-bc35a2062895 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:25.981425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaadbd-5cb2-41ce-b10d-bc35a2062895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:25.982166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaadbd-5cb2-41ce-b10d-bc35a2062895 None admin] RBAC: Adding query filter params (role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=868e9fb5616543f6b193d6f1b79ffef3, user.id=16aab776604b46f6b9f706d4d684a6b9) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:25.982475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaadbd-5cb2-41ce-b10d-bc35a2062895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:25.984018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaadbd-5cb2-41ce-b10d-bc35a2062895 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:25.991167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 50/97] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:25 2022] GET /identity/v3/role_assignments?user.id=16aab776604b46f6b9f706d4d684a6b9&scope.project.id=868e9fb5616543f6b193d6f1b79ffef3&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:27.444264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aa837-35ef-4ff9-814a-49281522e17e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:27.444832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aa837-35ef-4ff9-814a-49281522e17e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:27.444832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aa837-35ef-4ff9-814a-49281522e17e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:27.446795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 48/98] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:27.465400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f5d6-65c1-4281-b580-de67c8ec0d92 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:27.465897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f5d6-65c1-4281-b580-de67c8ec0d92 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:27.465967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f5d6-65c1-4281-b580-de67c8ec0d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:27.486309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0372f5d6-65c1-4281-b580-de67c8ec0d92 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:27.487646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0372f5d6-65c1-4281-b580-de67c8ec0d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:27.566174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 51/99] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:27.608188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eff29-6a76-4c96-bbd6-03fc6dd6b20b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:27.608498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eff29-6a76-4c96-bbd6-03fc6dd6b20b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:27.608804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eff29-6a76-4c96-bbd6-03fc6dd6b20b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:27.610164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 49/100] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:27.615848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9091c18-7d9f-42bb-b989-71a1a4e19ba6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:27.616140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9091c18-7d9f-42bb-b989-71a1a4e19ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:27.616420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9091c18-7d9f-42bb-b989-71a1a4e19ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:27.627707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:27.643821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a9091c18-7d9f-42bb-b989-71a1a4e19ba6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:27.645031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a9091c18-7d9f-42bb-b989-71a1a4e19ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:27.731656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 52/101] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:28.008831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eff29-6a76-4c96-bbd6-03fc6dd6b20b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:28.148965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eff29-6a76-4c96-bbd6-03fc6dd6b20b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:28.194872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147d49f-1f09-4b2b-896e-9c97a3de1540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:28.195556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147d49f-1f09-4b2b-896e-9c97a3de1540 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.195839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147d49f-1f09-4b2b-896e-9c97a3de1540 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.196112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147d49f-1f09-4b2b-896e-9c97a3de1540 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.207164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147d49f-1f09-4b2b-896e-9c97a3de1540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e147d49f-1f09-4b2b-896e-9c97a3de1540 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:28.220872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:28.221821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:28.222558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 50/102] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:28.228413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9091c18-7d9f-42bb-b989-71a1a4e19ba6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:28.229737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147d49f-1f09-4b2b-896e-9c97a3de1540 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:28.256754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:28.360101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9091c18-7d9f-42bb-b989-71a1a4e19ba6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:28.430760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68798615-b886-417c-83a7-38370bff8765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:28.431433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798615-b886-417c-83a7-38370bff8765 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.431718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798615-b886-417c-83a7-38370bff8765 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.431998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798615-b886-417c-83a7-38370bff8765 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.432679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68798615-b886-417c-83a7-38370bff8765 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:28.432984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68798615-b886-417c-83a7-38370bff8765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:28.434641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68798615-b886-417c-83a7-38370bff8765 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:28.436628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:28.443584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147d49f-1f09-4b2b-896e-9c97a3de1540 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:28.445855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 53/103] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:28.448566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c945a-72d6-4adb-aeae-c22d6cfd2f3b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.449110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c945a-72d6-4adb-aeae-c22d6cfd2f3b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.449575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c945a-72d6-4adb-aeae-c22d6cfd2f3b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.451164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 54/104] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:28.457439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1b0d7-7e0d-40d8-b2d3-51cc1c8b6a19 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.457999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1b0d7-7e0d-40d8-b2d3-51cc1c8b6a19 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.458816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1b0d7-7e0d-40d8-b2d3-51cc1c8b6a19 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.459807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 55/105] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:28.460514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1b0d7-7e0d-40d8-b2d3-51cc1c8b6a19 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:28.495052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b1f49-2c10-45bd-97bb-0a1f7983cd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:28.496789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1f49-2c10-45bd-97bb-0a1f7983cd61 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.497274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1f49-2c10-45bd-97bb-0a1f7983cd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.497793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1f49-2c10-45bd-97bb-0a1f7983cd61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.499306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b1f49-2c10-45bd-97bb-0a1f7983cd61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:28.501011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b1f49-2c10-45bd-97bb-0a1f7983cd61 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:28.503159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 51/106] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:28.505099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6157e1-da2c-4c57-982c-f94ede47191d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.505599 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6157e1-da2c-4c57-982c-f94ede47191d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.506211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6157e1-da2c-4c57-982c-f94ede47191d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.526337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1b6157e1-da2c-4c57-982c-f94ede47191d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:28.527901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1b6157e1-da2c-4c57-982c-f94ede47191d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:28.572636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1b0d7-7e0d-40d8-b2d3-51cc1c8b6a19 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:28.605804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 52/107] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:28.613190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9d474-ca6f-40eb-97c3-4521a98875fc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.613711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9d474-ca6f-40eb-97c3-4521a98875fc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.614269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9d474-ca6f-40eb-97c3-4521a98875fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.623505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6993a24-f9c0-4a95-9838-dfad12aea4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:28.624696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6993a24-f9c0-4a95-9838-dfad12aea4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.625184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6993a24-f9c0-4a95-9838-dfad12aea4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.625660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6993a24-f9c0-4a95-9838-dfad12aea4f2 None admin] PATH_INFO: `/v3/users/dbbb3ca9b5a844cb97e77a8c1e44c8eb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.649591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0ee9d474-ca6f-40eb-97c3-4521a98875fc None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:28.651127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0ee9d474-ca6f-40eb-97c3-4521a98875fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:28.653153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6993a24-f9c0-4a95-9838-dfad12aea4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=dbbb3ca9b5a844cb97e77a8c1e44c8eb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:28.655908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6993a24-f9c0-4a95-9838-dfad12aea4f2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:28.659114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 56/108] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:28 2022] GET /identity/v3/users/dbbb3ca9b5a844cb97e77a8c1e44c8eb => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:28.663885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6993a24-f9c0-4a95-9838-dfad12aea4f2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:28.668230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:28.739919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:28.781523 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6993a24-f9c0-4a95-9838-dfad12aea4f2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:28.789480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 53/109] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:28.790620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9d474-ca6f-40eb-97c3-4521a98875fc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:28.848011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eac9d-023a-4530-a5e3-fedd97a62ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:28.848651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eac9d-023a-4530-a5e3-fedd97a62ae4 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.848934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eac9d-023a-4530-a5e3-fedd97a62ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.849208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eac9d-023a-4530-a5e3-fedd97a62ae4 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.850149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5eac9d-023a-4530-a5e3-fedd97a62ae4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:28.851728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5eac9d-023a-4530-a5e3-fedd97a62ae4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:28.909767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 57/110] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:04:28 2022] POST /identity/v3/projects => generated 302 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:28.915934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9d474-ca6f-40eb-97c3-4521a98875fc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:28.955169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f0964-6eec-4f73-b2fe-6360ce1ac2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:28.956234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f0964-6eec-4f73-b2fe-6360ce1ac2db None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:28.956713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f0964-6eec-4f73-b2fe-6360ce1ac2db None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:28.957184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f0964-6eec-4f73-b2fe-6360ce1ac2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:28.973559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:29.035502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f0964-6eec-4f73-b2fe-6360ce1ac2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:29.037475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f0964-6eec-4f73-b2fe-6360ce1ac2db None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:29.146760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 54/111] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:29.154771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eac9d-023a-4530-a5e3-fedd97a62ae4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:29.157197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f0964-6eec-4f73-b2fe-6360ce1ac2db None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:29.215588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:29.278048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f0964-6eec-4f73-b2fe-6360ce1ac2db None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:29.280189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eac9d-023a-4530-a5e3-fedd97a62ae4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:29.288571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:29.314595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff9775-9ab8-4b2d-94c6-2120485ec140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:29.315263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff9775-9ab8-4b2d-94c6-2120485ec140 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.315555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff9775-9ab8-4b2d-94c6-2120485ec140 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.315834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff9775-9ab8-4b2d-94c6-2120485ec140 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.317891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446314a6-1ecb-4137-be31-5014a3fed9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:29.318539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446314a6-1ecb-4137-be31-5014a3fed9aa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.318818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446314a6-1ecb-4137-be31-5014a3fed9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.319091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446314a6-1ecb-4137-be31-5014a3fed9aa None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.320268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446314a6-1ecb-4137-be31-5014a3fed9aa None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:29.321741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446314a6-1ecb-4137-be31-5014a3fed9aa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:29.323619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 58/112] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:29 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:29.326871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff9775-9ab8-4b2d-94c6-2120485ec140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:29.329358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b4f4-b576-43f8-afcb-173b52a24cde None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.329689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b4f4-b576-43f8-afcb-173b52a24cde None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.330123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b4f4-b576-43f8-afcb-173b52a24cde None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.331603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 59/113] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-73ff9775-9ab8-4b2d-94c6-2120485ec140 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:29.332443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:29.333569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 55/114] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:04:29.339783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff9775-9ab8-4b2d-94c6-2120485ec140 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:29.340452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993874e9-394e-438c-8786-775bac3b7551 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.340452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993874e9-394e-438c-8786-775bac3b7551 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.340452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993874e9-394e-438c-8786-775bac3b7551 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.342730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 60/115] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:29.343595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993874e9-394e-438c-8786-775bac3b7551 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:29.446663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff9775-9ab8-4b2d-94c6-2120485ec140 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:29.448904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:29.456531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993874e9-394e-438c-8786-775bac3b7551 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:29.483917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a47f1-ce40-4c85-b347-dd76822a0a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:29.484564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a47f1-ce40-4c85-b347-dd76822a0a78 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.484844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a47f1-ce40-4c85-b347-dd76822a0a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.485117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a47f1-ce40-4c85-b347-dd76822a0a78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.496164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a47f1-ce40-4c85-b347-dd76822a0a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:29.500205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23c543-d64f-423c-8dce-e64a7b25d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:29.500838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c543-d64f-423c-8dce-e64a7b25d2ee None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.501118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c543-d64f-423c-8dce-e64a7b25d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.501389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c543-d64f-423c-8dce-e64a7b25d2ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.502261 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23c543-d64f-423c-8dce-e64a7b25d2ee None admin] RBAC: Adding query filter params (role.id=5681843b3fe64a0ebe56433cfca360e7, scope.project.id=de5700cfe30946bd8e0082d5b4964f98, user.id=dbbb3ca9b5a844cb97e77a8c1e44c8eb) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:29.502628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23c543-d64f-423c-8dce-e64a7b25d2ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c17a47f1-ce40-4c85-b347-dd76822a0a78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:29.503194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:29.504293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:29.505217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:29.505873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23c543-d64f-423c-8dce-e64a7b25d2ee None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:29.507365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 56/116] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:29.508386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a47f1-ce40-4c85-b347-dd76822a0a78 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:29.511308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 61/117] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:29 2022] GET /identity/v3/role_assignments?user.id=dbbb3ca9b5a844cb97e77a8c1e44c8eb&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=5681843b3fe64a0ebe56433cfca360e7 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:29.512486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a5a7-b7a2-4c5e-96da-39260d7e97de None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.512760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a5a7-b7a2-4c5e-96da-39260d7e97de None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.513033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a5a7-b7a2-4c5e-96da-39260d7e97de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.533519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1261a5a7-b7a2-4c5e-96da-39260d7e97de None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:29.534807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1261a5a7-b7a2-4c5e-96da-39260d7e97de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:29.577530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:29.645526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 62/118] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:29.647216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8a0e-3aa5-490a-8a97-21ab103a1da9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.647691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8a0e-3aa5-490a-8a97-21ab103a1da9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.648100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8a0e-3aa5-490a-8a97-21ab103a1da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.661697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a47f1-ce40-4c85-b347-dd76822a0a78 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:29.670884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7a3e8a0e-3aa5-490a-8a97-21ab103a1da9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:29.672369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7a3e8a0e-3aa5-490a-8a97-21ab103a1da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:29.723084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:29.723734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:29.724012 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:29.724280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:29.725019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:29.725634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:29.727823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:29.742053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 57/119] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:29.747698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:29.782513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 63/120] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:29.785474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e8a0e-3aa5-490a-8a97-21ab103a1da9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:29.820290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:29.970093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d52a06-6060-41af-8d69-b1ffe7d67546 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:29.991339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:29.995116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e8a0e-3aa5-490a-8a97-21ab103a1da9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.029848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.030710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.031151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.031576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.032381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:30.032915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:30.034610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:30.048917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 58/121] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:30.054523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.072006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a6c55-d4ab-4381-b61d-4034d1b6f772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.072669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a6c55-d4ab-4381-b61d-4034d1b6f772 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.073041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a6c55-d4ab-4381-b61d-4034d1b6f772 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.073216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a6c55-d4ab-4381-b61d-4034d1b6f772 None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.075804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a6c55-d4ab-4381-b61d-4034d1b6f772 None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:30.077271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a6c55-d4ab-4381-b61d-4034d1b6f772 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:30.078910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 64/122] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:29 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:30.079761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a6c55-d4ab-4381-b61d-4034d1b6f772 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.154617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:30.167221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411800e-c6a1-499b-b4be-3c6b8737a300 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.206978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12786-45aa-477a-9189-4b30e4a56e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.211077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12786-45aa-477a-9189-4b30e4a56e2d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.211605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12786-45aa-477a-9189-4b30e4a56e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.212013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12786-45aa-477a-9189-4b30e4a56e2d None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.215108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf12786-45aa-477a-9189-4b30e4a56e2d None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:30.216843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf12786-45aa-477a-9189-4b30e4a56e2d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:30.220801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 59/123] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:30 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:30.224366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12786-45aa-477a-9189-4b30e4a56e2d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.286179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a6c55-d4ab-4381-b61d-4034d1b6f772 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.334801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18401a9a-7a11-4242-887d-fff55208b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.336369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401a9a-7a11-4242-887d-fff55208b1ae None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.336944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401a9a-7a11-4242-887d-fff55208b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.337538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18401a9a-7a11-4242-887d-fff55208b1ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.339168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12786-45aa-477a-9189-4b30e4a56e2d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.352418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18401a9a-7a11-4242-887d-fff55208b1ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-18401a9a-7a11-4242-887d-fff55208b1ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:30.359933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:30.360800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:30.361627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:30.364064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 65/124] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:30.365273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18401a9a-7a11-4242-887d-fff55208b1ae None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.382733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5596046-c7c9-46da-a00c-b414cd0483a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.383679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596046-c7c9-46da-a00c-b414cd0483a3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.383962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596046-c7c9-46da-a00c-b414cd0483a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.384235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596046-c7c9-46da-a00c-b414cd0483a3 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.385594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5596046-c7c9-46da-a00c-b414cd0483a3 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:30.387764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5596046-c7c9-46da-a00c-b414cd0483a3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:30.390004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 60/125] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:30 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:30.390758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5596046-c7c9-46da-a00c-b414cd0483a3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.408214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:30.426500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:30.488470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18401a9a-7a11-4242-887d-fff55208b1ae None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.505608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5596046-c7c9-46da-a00c-b414cd0483a3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.530855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba413ff-5969-4c98-bbe3-66e34ca100a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.531561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba413ff-5969-4c98-bbe3-66e34ca100a7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.531843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba413ff-5969-4c98-bbe3-66e34ca100a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.532114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba413ff-5969-4c98-bbe3-66e34ca100a7 None admin] PATH_INFO: `/v3/projects/868e9fb5616543f6b193d6f1b79ffef3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.533434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba413ff-5969-4c98-bbe3-66e34ca100a7 None admin] RBAC: Authorizing `identity:get_project(project_id=868e9fb5616543f6b193d6f1b79ffef3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:30.534983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba413ff-5969-4c98-bbe3-66e34ca100a7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:30.536832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 66/126] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:30 2022] GET /identity/v3/projects/868e9fb5616543f6b193d6f1b79ffef3 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:30.537573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba413ff-5969-4c98-bbe3-66e34ca100a7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.546931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eba82f-87c6-4e27-acab-924ab36b1235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.547730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba82f-87c6-4e27-acab-924ab36b1235 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.548087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba82f-87c6-4e27-acab-924ab36b1235 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.548499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba82f-87c6-4e27-acab-924ab36b1235 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.559694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eba82f-87c6-4e27-acab-924ab36b1235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a3eba82f-87c6-4e27-acab-924ab36b1235 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:30.565291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:30.566985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:30.568873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:30.572815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 61/127] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:04:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:30.572815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eba82f-87c6-4e27-acab-924ab36b1235 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.641477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:30.650709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba413ff-5969-4c98-bbe3-66e34ca100a7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.663064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:30.681395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eba82f-87c6-4e27-acab-924ab36b1235 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.692071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.693028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.693469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.693871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.694679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:30.695205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:30.696919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:30.711388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 67/128] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:30.712636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.718380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b934be-2381-42ac-b5f5-b5ee37e023ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.719061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b934be-2381-42ac-b5f5-b5ee37e023ef None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.719374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b934be-2381-42ac-b5f5-b5ee37e023ef None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.719652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b934be-2381-42ac-b5f5-b5ee37e023ef None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98/users/16aab776604b46f6b9f706d4d684a6b9/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.721818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b934be-2381-42ac-b5f5-b5ee37e023ef None admin] RBAC: Authorizing `identity:create_grant(project_id=de5700cfe30946bd8e0082d5b4964f98, user_id=16aab776604b46f6b9f706d4d684a6b9, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:30.723347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b934be-2381-42ac-b5f5-b5ee37e023ef None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:30.742734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 62/129] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:30 2022] PUT /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98/users/16aab776604b46f6b9f706d4d684a6b9/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:30.747379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b934be-2381-42ac-b5f5-b5ee37e023ef None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:30.908250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb6051-6051-46d7-8c65-f061f7c79fe9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.921755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b934be-2381-42ac-b5f5-b5ee37e023ef None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:30.949112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717ff7-0603-4310-8139-c1baaa005c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:30.950398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717ff7-0603-4310-8139-c1baaa005c7f None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:30.950684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717ff7-0603-4310-8139-c1baaa005c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:30.951012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717ff7-0603-4310-8139-c1baaa005c7f None admin] PATH_INFO: `/v3/projects/868e9fb5616543f6b193d6f1b79ffef3/users/16aab776604b46f6b9f706d4d684a6b9/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:30.953397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8717ff7-0603-4310-8139-c1baaa005c7f None admin] RBAC: Authorizing `identity:create_grant(project_id=868e9fb5616543f6b193d6f1b79ffef3, user_id=16aab776604b46f6b9f706d4d684a6b9, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:30.954926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8717ff7-0603-4310-8139-c1baaa005c7f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:30.976222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 68/130] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:30 2022] PUT /identity/v3/projects/868e9fb5616543f6b193d6f1b79ffef3/users/16aab776604b46f6b9f706d4d684a6b9/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:30.977060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717ff7-0603-4310-8139-c1baaa005c7f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:31.001994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:31.002674 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.002926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.003195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.003911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:31.004215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:31.004511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:31.005681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:31.017444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 63/131] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:04:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:31.023535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:31.035499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:31.121147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717ff7-0603-4310-8139-c1baaa005c7f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:31.156371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b1823-9d19-419d-8be1-bcc660c4391f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:31.169710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d579a-dd28-4ce5-872c-a2fd73ad98ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:31.170629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d579a-dd28-4ce5-872c-a2fd73ad98ff None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.170911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d579a-dd28-4ce5-872c-a2fd73ad98ff None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.171184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d579a-dd28-4ce5-872c-a2fd73ad98ff None admin] PATH_INFO: `/v3/users/186838d97c1140bb9bd1a7296d1414bf` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.172987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d579a-dd28-4ce5-872c-a2fd73ad98ff None admin] RBAC: Authorizing `identity:get_user(user_id=186838d97c1140bb9bd1a7296d1414bf)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:31.174937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d579a-dd28-4ce5-872c-a2fd73ad98ff None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:31.177103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 69/132] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:30 2022] GET /identity/v3/users/186838d97c1140bb9bd1a7296d1414bf => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:31.178299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc175c0-edd6-45d4-965d-039ec4a5ecae None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.178580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc175c0-edd6-45d4-965d-039ec4a5ecae None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.178912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc175c0-edd6-45d4-965d-039ec4a5ecae None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.180746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 70/133] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:31.183952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc175c0-edd6-45d4-965d-039ec4a5ecae None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:31.195964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77081974-2b61-4a0b-a923-f8577c21734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:31.196744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77081974-2b61-4a0b-a923-f8577c21734b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.197088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77081974-2b61-4a0b-a923-f8577c21734b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.197460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77081974-2b61-4a0b-a923-f8577c21734b None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.198755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77081974-2b61-4a0b-a923-f8577c21734b None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:31.200352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77081974-2b61-4a0b-a923-f8577c21734b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:31.202092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 64/134] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:31 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:31.203564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabe45b-8640-4504-b8d8-93218c2ac6bd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.203960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabe45b-8640-4504-b8d8-93218c2ac6bd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.204294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabe45b-8640-4504-b8d8-93218c2ac6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.227178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5eabe45b-8640-4504-b8d8-93218c2ac6bd None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:31.228597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5eabe45b-8640-4504-b8d8-93218c2ac6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:31.287219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:31.306581 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc175c0-edd6-45d4-965d-039ec4a5ecae None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:31.310332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 65/135] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:31.311204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabe45b-8640-4504-b8d8-93218c2ac6bd None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:31.315692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:31.344483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693dedb-6ad9-4519-95ef-fe2d93291573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:31.345206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693dedb-6ad9-4519-95ef-fe2d93291573 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.345470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693dedb-6ad9-4519-95ef-fe2d93291573 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.345781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693dedb-6ad9-4519-95ef-fe2d93291573 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.346957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693dedb-6ad9-4519-95ef-fe2d93291573 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:31.348459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693dedb-6ad9-4519-95ef-fe2d93291573 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:31.350736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 71/136] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:31 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:31.356394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693dedb-6ad9-4519-95ef-fe2d93291573 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:31.440628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabe45b-8640-4504-b8d8-93218c2ac6bd None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:31.481728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42069532-4b44-49d8-8848-20addb450c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:31.482716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42069532-4b44-49d8-8848-20addb450c50 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.483184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42069532-4b44-49d8-8848-20addb450c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.483700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42069532-4b44-49d8-8848-20addb450c50 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.485014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693dedb-6ad9-4519-95ef-fe2d93291573 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:31.485351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42069532-4b44-49d8-8848-20addb450c50 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:31.487454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42069532-4b44-49d8-8848-20addb450c50 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:31.489676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 66/137] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:31 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:31.495321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42069532-4b44-49d8-8848-20addb450c50 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:31.521315 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24bcf8-6681-4ee1-bc75-5c5e90602041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:31.521980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bcf8-6681-4ee1-bc75-5c5e90602041 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.522262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bcf8-6681-4ee1-bc75-5c5e90602041 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.522534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bcf8-6681-4ee1-bc75-5c5e90602041 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98/users/186838d97c1140bb9bd1a7296d1414bf/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.524487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24bcf8-6681-4ee1-bc75-5c5e90602041 None admin] RBAC: Authorizing `identity:create_grant(project_id=de5700cfe30946bd8e0082d5b4964f98, user_id=186838d97c1140bb9bd1a7296d1414bf, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:31.525941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24bcf8-6681-4ee1-bc75-5c5e90602041 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:31.545904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 72/138] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:31 2022] PUT /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98/users/186838d97c1140bb9bd1a7296d1414bf/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:31.609260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:31.651182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42069532-4b44-49d8-8848-20addb450c50 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:31.694006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61621432-b0b2-4dca-b41a-41618cc40bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:31.694622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621432-b0b2-4dca-b41a-41618cc40bf5 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:31.694866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621432-b0b2-4dca-b41a-41618cc40bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:31.695130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621432-b0b2-4dca-b41a-41618cc40bf5 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98/users/16aab776604b46f6b9f706d4d684a6b9/roles/0f4eb127111b4fdabec35268db5bde9e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:31.697379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61621432-b0b2-4dca-b41a-41618cc40bf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=de5700cfe30946bd8e0082d5b4964f98, user_id=16aab776604b46f6b9f706d4d684a6b9, role_id=0f4eb127111b4fdabec35268db5bde9e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:31.698882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61621432-b0b2-4dca-b41a-41618cc40bf5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:31.715318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 67/139] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:31 2022] PUT /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98/users/16aab776604b46f6b9f706d4d684a6b9/roles/0f4eb127111b4fdabec35268db5bde9e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:31.875733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24bcf8-6681-4ee1-bc75-5c5e90602041 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:31.896333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:31.999487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24bcf8-6681-4ee1-bc75-5c5e90602041 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:32.041819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47f05f-8624-4ccc-b4f9-58e90802876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:32.042475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f05f-8624-4ccc-b4f9-58e90802876f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:32.042797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f05f-8624-4ccc-b4f9-58e90802876f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:32.043071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f05f-8624-4ccc-b4f9-58e90802876f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:32.044252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47f05f-8624-4ccc-b4f9-58e90802876f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:32.045714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47f05f-8624-4ccc-b4f9-58e90802876f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:32.047305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 73/140] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:32.053215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61621432-b0b2-4dca-b41a-41618cc40bf5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:32.185694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61621432-b0b2-4dca-b41a-41618cc40bf5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:32.219896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:32.221824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb786a-a297-4dcb-ac8e-d26e3a7415c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:32.222594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb786a-a297-4dcb-ac8e-d26e3a7415c2 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:32.222870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb786a-a297-4dcb-ac8e-d26e3a7415c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:32.223141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb786a-a297-4dcb-ac8e-d26e3a7415c2 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:32.223845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb786a-a297-4dcb-ac8e-d26e3a7415c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:32.225835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb786a-a297-4dcb-ac8e-d26e3a7415c2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:32.250974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 68/141] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:04:32 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:32.288422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4034c69-5dee-4bdc-b86f-b43648d1d323 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:32.288757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4034c69-5dee-4bdc-b86f-b43648d1d323 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:32.288968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4034c69-5dee-4bdc-b86f-b43648d1d323 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:32.290588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 74/142] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:32.296729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f7cd0-1faa-456b-8df0-65c93248bcc3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:32.297021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f7cd0-1faa-456b-8df0-65c93248bcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:32.297288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f7cd0-1faa-456b-8df0-65c93248bcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:32.319623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b86f7cd0-1faa-456b-8df0-65c93248bcc3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:32.320886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b86f7cd0-1faa-456b-8df0-65c93248bcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:32.395590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 69/143] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:32.676221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4034c69-5dee-4bdc-b86f-b43648d1d323 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:32.779630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:32.803166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4034c69-5dee-4bdc-b86f-b43648d1d323 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:32.844923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480a0de-d087-4e8f-936c-e446403aa911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:32.845688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480a0de-d087-4e8f-936c-e446403aa911 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:32.845963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480a0de-d087-4e8f-936c-e446403aa911 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:32.846231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480a0de-d087-4e8f-936c-e446403aa911 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:32.859507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480a0de-d087-4e8f-936c-e446403aa911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3480a0de-d087-4e8f-936c-e446403aa911 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:32.864344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:32.865643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:32.866918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 75/144] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:32.876232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f7cd0-1faa-456b-8df0-65c93248bcc3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:33.020459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f7cd0-1faa-456b-8df0-65c93248bcc3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:33.041067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:33.064136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:33.064653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:33.064934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:33.065210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:33.065893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:33.066198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:33.067663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:33.081578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 70/145] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:33.086811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480a0de-d087-4e8f-936c-e446403aa911 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:33.219051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480a0de-d087-4e8f-936c-e446403aa911 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:33.260266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d3c09-acdb-4943-b941-019ed96f78ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:33.260902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3c09-acdb-4943-b941-019ed96f78ad None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:33.261179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3c09-acdb-4943-b941-019ed96f78ad None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:33.261441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3c09-acdb-4943-b941-019ed96f78ad None admin] PATH_INFO: `/v3/users/dbbb3ca9b5a844cb97e77a8c1e44c8eb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:33.262568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d3c09-acdb-4943-b941-019ed96f78ad None admin] RBAC: Authorizing `identity:get_user(user_id=dbbb3ca9b5a844cb97e77a8c1e44c8eb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:33.264044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d3c09-acdb-4943-b941-019ed96f78ad None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:33.266020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 76/146] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:33 2022] GET /identity/v3/users/dbbb3ca9b5a844cb97e77a8c1e44c8eb => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:33.271713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:33.401170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83634c-7571-4501-a404-f1e5d619639b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:33.436409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6ea4c-ef03-4dbf-84b6-572d6618f162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:33.437050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6ea4c-ef03-4dbf-84b6-572d6618f162 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:33.437323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6ea4c-ef03-4dbf-84b6-572d6618f162 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:33.437591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6ea4c-ef03-4dbf-84b6-572d6618f162 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:33.438762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6ea4c-ef03-4dbf-84b6-572d6618f162 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:33.440267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6ea4c-ef03-4dbf-84b6-572d6618f162 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:33.441920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 71/147] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:33 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:33.447624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d3c09-acdb-4943-b941-019ed96f78ad None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:33.459610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:33.563842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d3c09-acdb-4943-b941-019ed96f78ad None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:33.600013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92119691-996d-446f-acbf-1b41e4120c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:33.600993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119691-996d-446f-acbf-1b41e4120c44 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:33.601265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119691-996d-446f-acbf-1b41e4120c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:33.601530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119691-996d-446f-acbf-1b41e4120c44 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98/users/dbbb3ca9b5a844cb97e77a8c1e44c8eb/roles/5681843b3fe64a0ebe56433cfca360e7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:33.603500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92119691-996d-446f-acbf-1b41e4120c44 None admin] RBAC: Authorizing `identity:create_grant(project_id=de5700cfe30946bd8e0082d5b4964f98, user_id=dbbb3ca9b5a844cb97e77a8c1e44c8eb, role_id=5681843b3fe64a0ebe56433cfca360e7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:33.604976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92119691-996d-446f-acbf-1b41e4120c44 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:33.625711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 77/148] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:33 2022] PUT /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98/users/dbbb3ca9b5a844cb97e77a8c1e44c8eb/roles/5681843b3fe64a0ebe56433cfca360e7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:33.750765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba343-49e9-478a-be61-a1fdb270a494 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:33.751089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba343-49e9-478a-be61-a1fdb270a494 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:33.751391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba343-49e9-478a-be61-a1fdb270a494 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:33.753018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 72/149] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:33.761765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fceba2-f53d-4665-808a-4cb3b3e69ec0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:33.762073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fceba2-f53d-4665-808a-4cb3b3e69ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:33.762310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fceba2-f53d-4665-808a-4cb3b3e69ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:33.785255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c6fceba2-f53d-4665-808a-4cb3b3e69ec0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:33.786463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c6fceba2-f53d-4665-808a-4cb3b3e69ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:33.864639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:33.877073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 78/150] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:34.195256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4a810-bf23-49e8-8146-b9b16e62a6fe None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:34.195735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4a810-bf23-49e8-8146-b9b16e62a6fe None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:34.195735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4a810-bf23-49e8-8146-b9b16e62a6fe None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:34.197062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 73/151] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:34.202272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b44d-e415-459a-a6eb-d7876f35bf47 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:34.202681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b44d-e415-459a-a6eb-d7876f35bf47 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:34.202870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b44d-e415-459a-a6eb-d7876f35bf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:34.224226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7cd6b44d-e415-459a-a6eb-d7876f35bf47 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:34.225712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7cd6b44d-e415-459a-a6eb-d7876f35bf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:34.315210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 79/152] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:34.339280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4a810-bf23-49e8-8146-b9b16e62a6fe None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:34.395089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:34.450979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4a810-bf23-49e8-8146-b9b16e62a6fe None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:34.487499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab026474-755b-42cd-aadc-26f8e40eea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:34.488139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab026474-755b-42cd-aadc-26f8e40eea6c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:34.488411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab026474-755b-42cd-aadc-26f8e40eea6c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:34.488676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab026474-755b-42cd-aadc-26f8e40eea6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:34.499621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab026474-755b-42cd-aadc-26f8e40eea6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:34.506162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5b06-714c-4fb9-a102-04a5c20e535c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:34.506438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5b06-714c-4fb9-a102-04a5c20e535c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:34.506707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5b06-714c-4fb9-a102-04a5c20e535c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:34.508367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 80/153] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ab026474-755b-42cd-aadc-26f8e40eea6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:34.508606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:34.509433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:34.510637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 74/154] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:34.515390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe5b06-714c-4fb9-a102-04a5c20e535c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:34.517685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6d89-236b-4aeb-bba9-009c7c33df79 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:34.517970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6d89-236b-4aeb-bba9-009c7c33df79 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:34.518233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6d89-236b-4aeb-bba9-009c7c33df79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:34.544686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-227f6d89-236b-4aeb-bba9-009c7c33df79 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:34.545884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-227f6d89-236b-4aeb-bba9-009c7c33df79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:34.657648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 75/155] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:34.721502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe5b06-714c-4fb9-a102-04a5c20e535c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:34.742867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6d89-236b-4aeb-bba9-009c7c33df79 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:34.758803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:34.774630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:34.775265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:34.775543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:34.775805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:34.776473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:34.776765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:34.778155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:34.789125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:34.797196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 81/156] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:34.804293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:34.858047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6d89-236b-4aeb-bba9-009c7c33df79 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:34.909355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a7322-1121-4c51-987b-a2df9e6cd2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:34.910663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a7322-1121-4c51-987b-a2df9e6cd2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:34.911290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a7322-1121-4c51-987b-a2df9e6cd2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:34.912017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a7322-1121-4c51-987b-a2df9e6cd2f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:34.929153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a7322-1121-4c51-987b-a2df9e6cd2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-df2a7322-1121-4c51-987b-a2df9e6cd2f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:34.938496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:34.939909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:34.940755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:34.943262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 76/157] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:04:34.950418 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a7322-1121-4c51-987b-a2df9e6cd2f5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:35.026531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd62468-5096-4440-8184-fd8d4f81c897 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:35.073352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:35.076390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a7322-1121-4c51-987b-a2df9e6cd2f5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:35.086675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd67356-c9a6-4740-ad74-9e33948077e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:35.091251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67356-c9a6-4740-ad74-9e33948077e1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:35.091251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67356-c9a6-4740-ad74-9e33948077e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:35.091251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67356-c9a6-4740-ad74-9e33948077e1 None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:35.091251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd67356-c9a6-4740-ad74-9e33948077e1 None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:35.091251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd67356-c9a6-4740-ad74-9e33948077e1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:35.092134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 82/158] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:34 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:35.100415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd67356-c9a6-4740-ad74-9e33948077e1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:35.150522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:35.151134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:35.151432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:35.151701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:35.152368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:35.152664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:35.154073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:35.178320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 77/159] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:35.184070 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:35.214502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:35.309335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd67356-c9a6-4740-ad74-9e33948077e1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:35.369056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144410d4-a3d4-43f2-9475-c0f097c50717 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:35.397058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1025ee-bc2e-488b-a8c6-418fc7eb5cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:35.397697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1025ee-bc2e-488b-a8c6-418fc7eb5cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:35.397977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1025ee-bc2e-488b-a8c6-418fc7eb5cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:35.398246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1025ee-bc2e-488b-a8c6-418fc7eb5cc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:35.448587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f941ef-4797-4d1a-8559-847d338f9118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:35.449247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f941ef-4797-4d1a-8559-847d338f9118 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:35.449522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f941ef-4797-4d1a-8559-847d338f9118 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:35.449794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f941ef-4797-4d1a-8559-847d338f9118 None admin] PATH_INFO: `/v3/users/186838d97c1140bb9bd1a7296d1414bf` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:35.456037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f941ef-4797-4d1a-8559-847d338f9118 None admin] RBAC: Authorizing `identity:get_user(user_id=186838d97c1140bb9bd1a7296d1414bf)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:35.457536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f941ef-4797-4d1a-8559-847d338f9118 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:35.459190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 78/160] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:35 2022] GET /identity/v3/users/186838d97c1140bb9bd1a7296d1414bf => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:35.459995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f941ef-4797-4d1a-8559-847d338f9118 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:35.529302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:35.541157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1025ee-bc2e-488b-a8c6-418fc7eb5cc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:35.542733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1025ee-bc2e-488b-a8c6-418fc7eb5cc0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:35.639548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:35.649983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f941ef-4797-4d1a-8559-847d338f9118 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:35.700745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce1543-6615-4a94-96e4-60f5486f775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:35.701455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce1543-6615-4a94-96e4-60f5486f775e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:35.701732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce1543-6615-4a94-96e4-60f5486f775e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:35.702000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce1543-6615-4a94-96e4-60f5486f775e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:35.704425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 83/161] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:35.705382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1025ee-bc2e-488b-a8c6-418fc7eb5cc0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:35.724255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce1543-6615-4a94-96e4-60f5486f775e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-99ce1543-6615-4a94-96e4-60f5486f775e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:35.730078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:35.730982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:35.731784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 79/162] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:35.736871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864ba9e-f51e-4d04-a05f-ab2ac128ba08 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:35.737167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864ba9e-f51e-4d04-a05f-ab2ac128ba08 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:35.737439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864ba9e-f51e-4d04-a05f-ab2ac128ba08 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:35.738882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 80/163] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:35.740980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864ba9e-f51e-4d04-a05f-ab2ac128ba08 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:35.815209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1025ee-bc2e-488b-a8c6-418fc7eb5cc0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:35.875641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9396-5242-4752-9e93-53f68e72f95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:35.877133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9396-5242-4752-9e93-53f68e72f95c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:35.877416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9396-5242-4752-9e93-53f68e72f95c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:35.877730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9396-5242-4752-9e93-53f68e72f95c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:35.912545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:35.934180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:35.945103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864ba9e-f51e-4d04-a05f-ab2ac128ba08 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:35.980274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9396-5242-4752-9e93-53f68e72f95c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:35.981823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9396-5242-4752-9e93-53f68e72f95c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:36.045757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a199-3f41-4b58-8e8c-a0cf0e8c3244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:36.046404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a199-3f41-4b58-8e8c-a0cf0e8c3244 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.046680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a199-3f41-4b58-8e8c-a0cf0e8c3244 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.046949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a199-3f41-4b58-8e8c-a0cf0e8c3244 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.048213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24a199-3f41-4b58-8e8c-a0cf0e8c3244 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:36.049851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24a199-3f41-4b58-8e8c-a0cf0e8c3244 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:36.051915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 81/164] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:35 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:36.052763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a199-3f41-4b58-8e8c-a0cf0e8c3244 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:36.099664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 84/165] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:36.101272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a242b1-9dc8-423f-aa75-ffe35762f7f0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.101555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a242b1-9dc8-423f-aa75-ffe35762f7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.101827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a242b1-9dc8-423f-aa75-ffe35762f7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.140727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-11a242b1-9dc8-423f-aa75-ffe35762f7f0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:36.142021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-11a242b1-9dc8-423f-aa75-ffe35762f7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:36.195976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a199-3f41-4b58-8e8c-a0cf0e8c3244 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:36.258849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:36.259527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.259803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.260645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.261458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:36.261827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:36.263283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:36.275874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:36.280773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 82/166] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:36.281770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:36.309218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 85/167] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:36.309902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a242b1-9dc8-423f-aa75-ffe35762f7f0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:36.324528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:36.474626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a242b1-9dc8-423f-aa75-ffe35762f7f0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:36.495690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e7186-dfc2-4c14-b6c6-b86440ba21bd None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:36.537157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f612d0-4035-43e5-bfcc-a82efff62469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:36.537948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f612d0-4035-43e5-bfcc-a82efff62469 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.538238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f612d0-4035-43e5-bfcc-a82efff62469 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.538512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f612d0-4035-43e5-bfcc-a82efff62469 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.543034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f612d0-4035-43e5-bfcc-a82efff62469 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:36.543034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f612d0-4035-43e5-bfcc-a82efff62469 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:36.543208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 86/168] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:36 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:36.543912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f612d0-4035-43e5-bfcc-a82efff62469 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:36.547869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:36.549509 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.549996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.550466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.551437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] RBAC: Adding query filter params (role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=de5700cfe30946bd8e0082d5b4964f98, user.id=16aab776604b46f6b9f706d4d684a6b9) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:36.551950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:36.553682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:36.560265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 83/169] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:36 2022] GET /identity/v3/role_assignments?user.id=16aab776604b46f6b9f706d4d684a6b9&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:36.561324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:36.668320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:36.724440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:36.744652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f612d0-4035-43e5-bfcc-a82efff62469 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:36.759953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b770-c0ce-44cd-a206-617766d37a3c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:36.801358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc497a1-0a13-4198-a7a3-f66ecc6f344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:36.802085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc497a1-0a13-4198-a7a3-f66ecc6f344f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.802362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc497a1-0a13-4198-a7a3-f66ecc6f344f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.802633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc497a1-0a13-4198-a7a3-f66ecc6f344f None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.803900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc497a1-0a13-4198-a7a3-f66ecc6f344f None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:36.805368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc497a1-0a13-4198-a7a3-f66ecc6f344f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:36.811422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 87/170] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:36 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:36.812862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024d79-edb6-45c4-ba46-3bc6619c6de5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.813146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024d79-edb6-45c4-ba46-3bc6619c6de5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.813419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024d79-edb6-45c4-ba46-3bc6619c6de5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.815263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 88/171] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:36.816095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65024d79-edb6-45c4-ba46-3bc6619c6de5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:36.826522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71099c1-00e8-444a-b4f8-148d382e5aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:36.827149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71099c1-00e8-444a-b4f8-148d382e5aae None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.827580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71099c1-00e8-444a-b4f8-148d382e5aae None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.827855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71099c1-00e8-444a-b4f8-148d382e5aae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.828741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71099c1-00e8-444a-b4f8-148d382e5aae None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=de5700cfe30946bd8e0082d5b4964f98, user.id=186838d97c1140bb9bd1a7296d1414bf) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:36.829059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71099c1-00e8-444a-b4f8-148d382e5aae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:36.830689 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71099c1-00e8-444a-b4f8-148d382e5aae None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:36.845407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 84/172] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:36 2022] GET /identity/v3/role_assignments?user.id=186838d97c1140bb9bd1a7296d1414bf&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:36.846683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d693-8a8e-4396-b731-ef3b92b23463 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:36.846967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d693-8a8e-4396-b731-ef3b92b23463 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:36.847368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d693-8a8e-4396-b731-ef3b92b23463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:36.880362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cdc6d693-8a8e-4396-b731-ef3b92b23463 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:36.881729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cdc6d693-8a8e-4396-b731-ef3b92b23463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:36.964387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65024d79-edb6-45c4-ba46-3bc6619c6de5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:36.993033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:37.006113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf69b-f82b-4f8f-b0f6-7ab3682c8be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:37.006778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf69b-f82b-4f8f-b0f6-7ab3682c8be4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.007116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf69b-f82b-4f8f-b0f6-7ab3682c8be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.007496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf69b-f82b-4f8f-b0f6-7ab3682c8be4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.011823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 85/173] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:37.035817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6d693-8a8e-4396-b731-ef3b92b23463 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:37.080460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf69b-f82b-4f8f-b0f6-7ab3682c8be4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:37.081973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf69b-f82b-4f8f-b0f6-7ab3682c8be4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:37.170323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:37.190505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 89/174] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:37.196081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf69b-f82b-4f8f-b0f6-7ab3682c8be4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:37.213089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6d693-8a8e-4396-b731-ef3b92b23463 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:37.222079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:37.280909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d75421-b4b0-4aea-a6b4-c2306b3ff4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:37.281494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d75421-b4b0-4aea-a6b4-c2306b3ff4be None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.281887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d75421-b4b0-4aea-a6b4-c2306b3ff4be None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.282169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d75421-b4b0-4aea-a6b4-c2306b3ff4be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.304529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d75421-b4b0-4aea-a6b4-c2306b3ff4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:37.308163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf69b-f82b-4f8f-b0f6-7ab3682c8be4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c2d75421-b4b0-4aea-a6b4-c2306b3ff4be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:37.310007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:37.310936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:37.311796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 86/175] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:04:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:37.318754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d75421-b4b0-4aea-a6b4-c2306b3ff4be None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:37.345545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23366d9-857f-49e6-845b-05ecbbff68c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:37.346191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23366d9-857f-49e6-845b-05ecbbff68c5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.346504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23366d9-857f-49e6-845b-05ecbbff68c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.346780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23366d9-857f-49e6-845b-05ecbbff68c5 None admin] PATH_INFO: `/v3/projects/868e9fb5616543f6b193d6f1b79ffef3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.348031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23366d9-857f-49e6-845b-05ecbbff68c5 None admin] RBAC: Authorizing `identity:get_project(project_id=868e9fb5616543f6b193d6f1b79ffef3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:37.349603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23366d9-857f-49e6-845b-05ecbbff68c5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:37.352903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 90/176] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:37 2022] GET /identity/v3/projects/868e9fb5616543f6b193d6f1b79ffef3 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:37.354179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e933c8-4ce4-4521-bb95-d06f21185bf8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.354457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e933c8-4ce4-4521-bb95-d06f21185bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.354744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e933c8-4ce4-4521-bb95-d06f21185bf8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.360156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 91/177] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:37.361059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e933c8-4ce4-4521-bb95-d06f21185bf8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:37.458621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:37.472787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e933c8-4ce4-4521-bb95-d06f21185bf8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:37.474731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d75421-b4b0-4aea-a6b4-c2306b3ff4be None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:37.513615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8bd07-9820-40e6-a6a1-8c77d5e5564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:37.514100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:37.514719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.515002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.515333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.515558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8bd07-9820-40e6-a6a1-8c77d5e5564c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.516005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:37.516315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:37.516564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8bd07-9820-40e6-a6a1-8c77d5e5564c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.517021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8bd07-9820-40e6-a6a1-8c77d5e5564c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.517895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8bd07-9820-40e6-a6a1-8c77d5e5564c None admin] RBAC: Adding query filter params (role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=868e9fb5616543f6b193d6f1b79ffef3, user.id=16aab776604b46f6b9f706d4d684a6b9) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:37.518201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:37.518500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8bd07-9820-40e6-a6a1-8c77d5e5564c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:37.520233 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8bd07-9820-40e6-a6a1-8c77d5e5564c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:37.521759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:37.528213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 92/178] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:37 2022] GET /identity/v3/role_assignments?user.id=16aab776604b46f6b9f706d4d684a6b9&scope.project.id=868e9fb5616543f6b193d6f1b79ffef3&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:37.529854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57097066-66ad-4812-8d37-167b6a3e4170 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.530176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 87/179] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:04:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:37.530489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57097066-66ad-4812-8d37-167b6a3e4170 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.530972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57097066-66ad-4812-8d37-167b6a3e4170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.535894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:37.550819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-57097066-66ad-4812-8d37-167b6a3e4170 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:37.552466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-57097066-66ad-4812-8d37-167b6a3e4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:37.633380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 93/180] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:37.634746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57097066-66ad-4812-8d37-167b6a3e4170 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:37.782705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:37.787514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9dd0-366d-47fa-9b9e-d84888aa25ac None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:37.806606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57097066-66ad-4812-8d37-167b6a3e4170 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:37.846869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ab3ef-08c8-4479-b5df-4b98f06da43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:37.847603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab3ef-08c8-4479-b5df-4b98f06da43b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.847889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab3ef-08c8-4479-b5df-4b98f06da43b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.848169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab3ef-08c8-4479-b5df-4b98f06da43b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.862040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ab3ef-08c8-4479-b5df-4b98f06da43b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a90ab3ef-08c8-4479-b5df-4b98f06da43b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:37.867471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:37.868364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:37.869142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:37.870204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 94/181] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:37.871054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588c722-e6d5-40f9-aed3-aa8886d42a69 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.871364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588c722-e6d5-40f9-aed3-aa8886d42a69 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.871639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588c722-e6d5-40f9-aed3-aa8886d42a69 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.873597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 95/182] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:37.879567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d92c0-f6d2-492c-b138-8f4484f7c45c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.879852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d92c0-f6d2-492c-b138-8f4484f7c45c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.880124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d92c0-f6d2-492c-b138-8f4484f7c45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.906578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d535f4-0657-44e7-b611-60a706f72f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:37.908094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d535f4-0657-44e7-b611-60a706f72f53 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:37.908094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d535f4-0657-44e7-b611-60a706f72f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:37.908094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d535f4-0657-44e7-b611-60a706f72f53 None admin] PATH_INFO: `/v3/users/16aab776604b46f6b9f706d4d684a6b9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:37.909347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-265d92c0-f6d2-492c-b138-8f4484f7c45c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:37.910529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-265d92c0-f6d2-492c-b138-8f4484f7c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:37.912029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d535f4-0657-44e7-b611-60a706f72f53 None admin] RBAC: Authorizing `identity:get_user(user_id=16aab776604b46f6b9f706d4d684a6b9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:37.915201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d535f4-0657-44e7-b611-60a706f72f53 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:37.917562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 88/183] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:37 2022] GET /identity/v3/users/16aab776604b46f6b9f706d4d684a6b9 => generated 275 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:37.918714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d535f4-0657-44e7-b611-60a706f72f53 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:37.994138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 96/184] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:37.994999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d92c0-f6d2-492c-b138-8f4484f7c45c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:38.008180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:38.056648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d535f4-0657-44e7-b611-60a706f72f53 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.086931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:38.100643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.101544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.101826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.102098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.102928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:38.103563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:38.105369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:38.108470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d92c0-f6d2-492c-b138-8f4484f7c45c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.115863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 89/185] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:38.121123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:38.144868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed54f4-92dd-4a45-9a8d-5645d97f670e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.145475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed54f4-92dd-4a45-9a8d-5645d97f670e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.145751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed54f4-92dd-4a45-9a8d-5645d97f670e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.146021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed54f4-92dd-4a45-9a8d-5645d97f670e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.242442 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed54f4-92dd-4a45-9a8d-5645d97f670e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:38.244021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed54f4-92dd-4a45-9a8d-5645d97f670e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:38.258340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee23c8-ce33-4ea1-8bbb-fc4aae1c5868 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.288067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:38.296268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be55523-0b84-4706-9fb3-caa6690e6813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.296910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55523-0b84-4706-9fb3-caa6690e6813 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.297191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55523-0b84-4706-9fb3-caa6690e6813 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.297460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55523-0b84-4706-9fb3-caa6690e6813 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.300048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be55523-0b84-4706-9fb3-caa6690e6813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:38.301763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be55523-0b84-4706-9fb3-caa6690e6813 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:38.307928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 90/186] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:38.307928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6bffd-b6c8-4ea3-bc8f-f1400ce22ae7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.307928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6bffd-b6c8-4ea3-bc8f-f1400ce22ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.307928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6bffd-b6c8-4ea3-bc8f-f1400ce22ae7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.307928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 91/187] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:38.307928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6bffd-b6c8-4ea3-bc8f-f1400ce22ae7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:38.338229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:38.363645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 97/188] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:38.368598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed54f4-92dd-4a45-9a8d-5645d97f670e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:38.421785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6bffd-b6c8-4ea3-bc8f-f1400ce22ae7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.465220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d96b3-3fb4-4808-aa80-32ccaf5b584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.466314 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d96b3-3fb4-4808-aa80-32ccaf5b584a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.466589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d96b3-3fb4-4808-aa80-32ccaf5b584a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.467068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d96b3-3fb4-4808-aa80-32ccaf5b584a None admin] PATH_INFO: `/v3/users/6fe1e399f3be4a1495299fae0f0ca715` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.477952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed54f4-92dd-4a45-9a8d-5645d97f670e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.483760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d96b3-3fb4-4808-aa80-32ccaf5b584a None admin] RBAC: Authorizing `identity:get_user(user_id=6fe1e399f3be4a1495299fae0f0ca715)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:38.485344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d96b3-3fb4-4808-aa80-32ccaf5b584a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:38.486984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 92/189] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:38 2022] GET /identity/v3/users/6fe1e399f3be4a1495299fae0f0ca715 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:38.492410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d96b3-3fb4-4808-aa80-32ccaf5b584a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:38.516602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6302b-70a2-45bf-bfc4-666054961fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.517494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6302b-70a2-45bf-bfc4-666054961fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.517780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6302b-70a2-45bf-bfc4-666054961fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.518304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6302b-70a2-45bf-bfc4-666054961fc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.531035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6302b-70a2-45bf-bfc4-666054961fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fdb6302b-70a2-45bf-bfc4-666054961fc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:38.535958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:38.536894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:38.537710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 98/190] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:04:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:38.538866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ffcd-7ea3-46ec-b1ed-d0a9e0354c23 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.538866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ffcd-7ea3-46ec-b1ed-d0a9e0354c23 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.538866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ffcd-7ea3-46ec-b1ed-d0a9e0354c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.556850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9dc8ffcd-7ea3-46ec-b1ed-d0a9e0354c23 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:38.558091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9dc8ffcd-7ea3-46ec-b1ed-d0a9e0354c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:38.558627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:38.587245 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:38.603553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d96b3-3fb4-4808-aa80-32ccaf5b584a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.625591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 99/191] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:38.626467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8ffcd-7ea3-46ec-b1ed-d0a9e0354c23 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:38.637120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138288ff-d04c-44fb-9356-2d4d007d03e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.637733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138288ff-d04c-44fb-9356-2d4d007d03e5 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.638011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138288ff-d04c-44fb-9356-2d4d007d03e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.638279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138288ff-d04c-44fb-9356-2d4d007d03e5 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.638926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138288ff-d04c-44fb-9356-2d4d007d03e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:38.640422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138288ff-d04c-44fb-9356-2d4d007d03e5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:38.664019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 93/192] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:04:38 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:38.668609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138288ff-d04c-44fb-9356-2d4d007d03e5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:38.756729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8ffcd-7ea3-46ec-b1ed-d0a9e0354c23 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.796535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96675b3c-bdb9-4a74-bf81-a7999277dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.797223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96675b3c-bdb9-4a74-bf81-a7999277dde2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.797613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96675b3c-bdb9-4a74-bf81-a7999277dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.798004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96675b3c-bdb9-4a74-bf81-a7999277dde2 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.799427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96675b3c-bdb9-4a74-bf81-a7999277dde2 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:38.801030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96675b3c-bdb9-4a74-bf81-a7999277dde2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:38.807392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 100/193] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:38 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:38.807392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96675b3c-bdb9-4a74-bf81-a7999277dde2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:38.815410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138288ff-d04c-44fb-9356-2d4d007d03e5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.840176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:38.845130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:38.859741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5388c-5cbc-495e-8f73-c861536a9632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.861107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5388c-5cbc-495e-8f73-c861536a9632 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.861535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5388c-5cbc-495e-8f73-c861536a9632 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.863421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5388c-5cbc-495e-8f73-c861536a9632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.944760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96675b3c-bdb9-4a74-bf81-a7999277dde2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:38.952359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5388c-5cbc-495e-8f73-c861536a9632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:38.954399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5388c-5cbc-495e-8f73-c861536a9632 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:38.993644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:38.994566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:38.994893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:38.995172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:38.996158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:38.996464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:38.998035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:39.011719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 101/194] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:04:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:39.012853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.092306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 94/195] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:39.092306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5388c-5cbc-495e-8f73-c861536a9632 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.106428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:39.134569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:39.141069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00580ee5-dce5-418b-81ca-6bb5efbb4941 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:39.181624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:39.182566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:39.183000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:39.183481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:39.184347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] RBAC: Adding query filter params (role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=de5700cfe30946bd8e0082d5b4964f98, user.id=16aab776604b46f6b9f706d4d684a6b9) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:39.184793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:39.186602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:39.194604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 102/196] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/role_assignments?user.id=16aab776604b46f6b9f706d4d684a6b9&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:39.195597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.225957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5388c-5cbc-495e-8f73-c861536a9632 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:39.269695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614013-dee6-4d4c-a7dc-8f96dd2a3a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:39.270631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614013-dee6-4d4c-a7dc-8f96dd2a3a93 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:39.271039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614013-dee6-4d4c-a7dc-8f96dd2a3a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:39.271494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614013-dee6-4d4c-a7dc-8f96dd2a3a93 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:39.284192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41614013-dee6-4d4c-a7dc-8f96dd2a3a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-41614013-dee6-4d4c-a7dc-8f96dd2a3a93 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:39.289520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:39.290515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:39.291367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:39.293459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 95/197] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:39.294447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614013-dee6-4d4c-a7dc-8f96dd2a3a93 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.323161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fe112-ffa9-4d87-bae0-1ae2f1a6b262 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:39.372019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ba90f-b534-4e98-a539-328cdf4071cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:39.372670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba90f-b534-4e98-a539-328cdf4071cd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:39.372916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba90f-b534-4e98-a539-328cdf4071cd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:39.373191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba90f-b534-4e98-a539-328cdf4071cd None admin] PATH_INFO: `/v3/users/6fe1e399f3be4a1495299fae0f0ca715` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:39.379012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ba90f-b534-4e98-a539-328cdf4071cd None admin] RBAC: Authorizing `identity:get_user(user_id=6fe1e399f3be4a1495299fae0f0ca715)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:39.379012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ba90f-b534-4e98-a539-328cdf4071cd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:39.379012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 103/198] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/users/6fe1e399f3be4a1495299fae0f0ca715 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:39.381015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ba90f-b534-4e98-a539-328cdf4071cd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.389163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:39.426439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:39.444755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614013-dee6-4d4c-a7dc-8f96dd2a3a93 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:39.490903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956f7bc-e3d8-4f66-b506-6acc1f715d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:39.492295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956f7bc-e3d8-4f66-b506-6acc1f715d8c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:39.492783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956f7bc-e3d8-4f66-b506-6acc1f715d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:39.493246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956f7bc-e3d8-4f66-b506-6acc1f715d8c None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:39.495139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956f7bc-e3d8-4f66-b506-6acc1f715d8c None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:39.497138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956f7bc-e3d8-4f66-b506-6acc1f715d8c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:39.499847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 96/199] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:39.500995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956f7bc-e3d8-4f66-b506-6acc1f715d8c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.512690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ba90f-b534-4e98-a539-328cdf4071cd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:39.559805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:39.560427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:39.560708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:39.560964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:39.561629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:39.561928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:39.563394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:39.573005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 104/200] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:39.573841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.642328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956f7bc-e3d8-4f66-b506-6acc1f715d8c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:39.688225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:39.691983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031175f2-3a91-4721-bab7-c9c0793ac764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:39.692641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031175f2-3a91-4721-bab7-c9c0793ac764 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:39.692948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031175f2-3a91-4721-bab7-c9c0793ac764 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:39.693186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031175f2-3a91-4721-bab7-c9c0793ac764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:39.702163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338bdd3-3b70-450f-9884-4230420d678e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:39.706715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:39.746032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:39.746995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:39.747492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:39.747938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:39.748842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374, user.id=6fe1e399f3be4a1495299fae0f0ca715) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:39.749854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:39.752418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:39.760994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 105/201] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/role_assignments?user.id=6fe1e399f3be4a1495299fae0f0ca715&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:39.762055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.809706 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031175f2-3a91-4721-bab7-c9c0793ac764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:39.811203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031175f2-3a91-4721-bab7-c9c0793ac764 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:39.886699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1c21b-a9cc-4751-8f28-52afe6fa6be9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:39.930822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8211ee-2221-4a89-ab37-58ef219bd789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:39.931815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8211ee-2221-4a89-ab37-58ef219bd789 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:39.932916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8211ee-2221-4a89-ab37-58ef219bd789 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:39.933369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8211ee-2221-4a89-ab37-58ef219bd789 None admin] PATH_INFO: `/v3/users/dbbb3ca9b5a844cb97e77a8c1e44c8eb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:39.935011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8211ee-2221-4a89-ab37-58ef219bd789 None admin] RBAC: Authorizing `identity:get_user(user_id=dbbb3ca9b5a844cb97e77a8c1e44c8eb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:39.936770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8211ee-2221-4a89-ab37-58ef219bd789 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:39.938703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 106/202] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/users/dbbb3ca9b5a844cb97e77a8c1e44c8eb => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:39.944354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8211ee-2221-4a89-ab37-58ef219bd789 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:39.999765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:40.039169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 97/203] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:40.046342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031175f2-3a91-4721-bab7-c9c0793ac764 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:40.096236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:40.161738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8211ee-2221-4a89-ab37-58ef219bd789 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:40.171355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031175f2-3a91-4721-bab7-c9c0793ac764 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:40.218969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b3339-bc12-4458-816b-d20bbd797d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:40.220175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3339-bc12-4458-816b-d20bbd797d99 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:40.220680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3339-bc12-4458-816b-d20bbd797d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:40.221342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3339-bc12-4458-816b-d20bbd797d99 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:40.222723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b3339-bc12-4458-816b-d20bbd797d99 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:40.225244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b3339-bc12-4458-816b-d20bbd797d99 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:40.227121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 98/204] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:40 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:40.236351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b3339-bc12-4458-816b-d20bbd797d99 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:40.263191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b62f2e-eae8-4fe4-bc65-b69b64af4038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:40.263831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b62f2e-eae8-4fe4-bc65-b69b64af4038 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:40.264110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b62f2e-eae8-4fe4-bc65-b69b64af4038 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:40.264630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b62f2e-eae8-4fe4-bc65-b69b64af4038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:40.333455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b62f2e-eae8-4fe4-bc65-b69b64af4038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:40.334969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b62f2e-eae8-4fe4-bc65-b69b64af4038 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:40.359877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:40.436938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 107/205] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:04:40.444279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b62f2e-eae8-4fe4-bc65-b69b64af4038 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:40.459459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b3339-bc12-4458-816b-d20bbd797d99 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:40.469349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:40.501341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:40.502385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:40.502871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:40.503381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:40.504307 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] RBAC: Adding query filter params (role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374, user.id=6fe1e399f3be4a1495299fae0f0ca715) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:40.505080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:40.507550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:40.514163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 99/206] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:40 2022] GET /identity/v3/role_assignments?user.id=6fe1e399f3be4a1495299fae0f0ca715&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:40.631328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b62f2e-eae8-4fe4-bc65-b69b64af4038 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:40.664988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:40.685064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3da9-a008-4f47-b0a2-e1cc9ea9853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:40.686122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3da9-a008-4f47-b0a2-e1cc9ea9853f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:40.686609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3da9-a008-4f47-b0a2-e1cc9ea9853f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:40.687162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3da9-a008-4f47-b0a2-e1cc9ea9853f None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:40.688726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3da9-a008-4f47-b0a2-e1cc9ea9853f None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:40.690708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3da9-a008-4f47-b0a2-e1cc9ea9853f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:40.692728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 108/207] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:40 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:40.698756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:40.823855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd070d-d4aa-46ba-869f-a1c12286ea5e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:40.861880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdbf17-8e09-4b6e-beab-b172d2b5ff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:40.862883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbf17-8e09-4b6e-beab-b172d2b5ff79 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:40.863424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbf17-8e09-4b6e-beab-b172d2b5ff79 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:40.863888 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdbf17-8e09-4b6e-beab-b172d2b5ff79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:40.864856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdbf17-8e09-4b6e-beab-b172d2b5ff79 None admin] RBAC: Adding query filter params (role.id=5681843b3fe64a0ebe56433cfca360e7, scope.project.id=de5700cfe30946bd8e0082d5b4964f98, user.id=dbbb3ca9b5a844cb97e77a8c1e44c8eb) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:40.865404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdbf17-8e09-4b6e-beab-b172d2b5ff79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:40.867065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdbf17-8e09-4b6e-beab-b172d2b5ff79 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:40.873899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 100/208] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:40 2022] GET /identity/v3/role_assignments?user.id=dbbb3ca9b5a844cb97e77a8c1e44c8eb&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=5681843b3fe64a0ebe56433cfca360e7 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:40.978152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac4ff5-fd4c-420d-bd96-553a1adbc82c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:40.978821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac4ff5-fd4c-420d-bd96-553a1adbc82c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:40.979347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac4ff5-fd4c-420d-bd96-553a1adbc82c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:40.980939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 109/209] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:40.987057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69532745-4c3c-49e1-b11f-f0abfe5ec14b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:40.987693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69532745-4c3c-49e1-b11f-f0abfe5ec14b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:40.988165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69532745-4c3c-49e1-b11f-f0abfe5ec14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:41.013358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-69532745-4c3c-49e1-b11f-f0abfe5ec14b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:41.015130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69532745-4c3c-49e1-b11f-f0abfe5ec14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:41.046486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:41.092462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 101/210] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:41.166440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefa96-0d91-4bd8-b838-50bf7a2004c4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:41.167068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefa96-0d91-4bd8-b838-50bf7a2004c4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:41.167626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefa96-0d91-4bd8-b838-50bf7a2004c4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:41.169290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 110/211] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:41.174989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832a654-51fa-4653-8f8e-37cb53e8a4b0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:41.175603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832a654-51fa-4653-8f8e-37cb53e8a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:41.176077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832a654-51fa-4653-8f8e-37cb53e8a4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:41.200592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d832a654-51fa-4653-8f8e-37cb53e8a4b0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:41.202151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d832a654-51fa-4653-8f8e-37cb53e8a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:41.298485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 102/212] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:41.375107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aefa96-0d91-4bd8-b838-50bf7a2004c4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:41.482561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aefa96-0d91-4bd8-b838-50bf7a2004c4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:41.516818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04238c5-eb1b-4c44-b27b-08a2c3958671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:41.517539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04238c5-eb1b-4c44-b27b-08a2c3958671 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:41.517818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04238c5-eb1b-4c44-b27b-08a2c3958671 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:41.518085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04238c5-eb1b-4c44-b27b-08a2c3958671 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:41.528854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04238c5-eb1b-4c44-b27b-08a2c3958671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b04238c5-eb1b-4c44-b27b-08a2c3958671 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:41.534475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:41.535422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:41.536247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 111/213] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:41.541511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832a654-51fa-4653-8f8e-37cb53e8a4b0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:41.580432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04238c5-eb1b-4c44-b27b-08a2c3958671 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:41.634071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:41.664058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832a654-51fa-4653-8f8e-37cb53e8a4b0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:41.679731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:41.687703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04238c5-eb1b-4c44-b27b-08a2c3958671 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:41.714794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:41.715471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:41.715725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:41.715994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:41.717102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:41.717409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:41.719134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:41.728972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756f48e-900f-4716-a516-81da1aa69fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:41.729959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f48e-900f-4716-a516-81da1aa69fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:41.730421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f48e-900f-4716-a516-81da1aa69fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:41.730889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f48e-900f-4716-a516-81da1aa69fe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:41.734756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 103/214] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:41.736791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:41.737344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756f48e-900f-4716-a516-81da1aa69fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:41.739121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756f48e-900f-4716-a516-81da1aa69fe2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:41.747453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 112/215] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:41.749163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756f48e-900f-4716-a516-81da1aa69fe2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:41.843681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f50c1-1770-4de8-8808-cf797874bf3b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:41.884571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756f48e-900f-4716-a516-81da1aa69fe2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:41.888733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef2640-cb0d-4b98-b6f9-63eed477dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:41.894349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef2640-cb0d-4b98-b6f9-63eed477dae7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:41.894349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef2640-cb0d-4b98-b6f9-63eed477dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:41.894651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef2640-cb0d-4b98-b6f9-63eed477dae7 None admin] PATH_INFO: `/v3/users/cc27c4ba7fc8495b91efdaf391ca4359` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:41.907340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:41.915083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef2640-cb0d-4b98-b6f9-63eed477dae7 None admin] RBAC: Authorizing `identity:get_user(user_id=cc27c4ba7fc8495b91efdaf391ca4359)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:41.916704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef2640-cb0d-4b98-b6f9-63eed477dae7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:41.918608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 104/216] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:41 2022] GET /identity/v3/users/cc27c4ba7fc8495b91efdaf391ca4359 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:41.924177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef2640-cb0d-4b98-b6f9-63eed477dae7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:41.930623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabea3f3-77bc-48ec-bbba-16dded581c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:41.931539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea3f3-77bc-48ec-bbba-16dded581c84 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:41.932012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea3f3-77bc-48ec-bbba-16dded581c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:41.932470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea3f3-77bc-48ec-bbba-16dded581c84 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:41.933801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabea3f3-77bc-48ec-bbba-16dded581c84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:41.936198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabea3f3-77bc-48ec-bbba-16dded581c84 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:41.940413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:41.966235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 113/217] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:04:41 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:04:42.047735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef2640-cb0d-4b98-b6f9-63eed477dae7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:42.055904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a597bd-36c1-4199-9912-63cf35134d47 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:42.056556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a597bd-36c1-4199-9912-63cf35134d47 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:42.057075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a597bd-36c1-4199-9912-63cf35134d47 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:42.059053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 114/218] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:42.069127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284db1-43eb-4904-a95b-1afc287ef458 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:42.069412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284db1-43eb-4904-a95b-1afc287ef458 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:42.069685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284db1-43eb-4904-a95b-1afc287ef458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:42.091018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f5e85-4fbc-44b5-9543-6dfa61efd5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:42.091481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c3284db1-43eb-4904-a95b-1afc287ef458 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:42.092212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f5e85-4fbc-44b5-9543-6dfa61efd5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:42.092664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f5e85-4fbc-44b5-9543-6dfa61efd5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:42.092793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c3284db1-43eb-4904-a95b-1afc287ef458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:42.093265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f5e85-4fbc-44b5-9543-6dfa61efd5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:42.172246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 115/219] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:42.174505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f5e85-4fbc-44b5-9543-6dfa61efd5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:42.176179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f5e85-4fbc-44b5-9543-6dfa61efd5a1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:42.192704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:42.287223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 105/220] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:04:42.293736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3284db1-43eb-4904-a95b-1afc287ef458 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:42.400528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3284db1-43eb-4904-a95b-1afc287ef458 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:42.402975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:42.438258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a2c2a-5f37-40bb-a9e6-5e63d70f48e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:42.438802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2c2a-5f37-40bb-a9e6-5e63d70f48e0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:42.439079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2c2a-5f37-40bb-a9e6-5e63d70f48e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:42.439396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2c2a-5f37-40bb-a9e6-5e63d70f48e0 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:42.440555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a2c2a-5f37-40bb-a9e6-5e63d70f48e0 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:42.442025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a2c2a-5f37-40bb-a9e6-5e63d70f48e0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:42.444085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 116/221] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:42 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:42.449807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f5e85-4fbc-44b5-9543-6dfa61efd5a1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:42.467379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a2c2a-5f37-40bb-a9e6-5e63d70f48e0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:42.591408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f5e85-4fbc-44b5-9543-6dfa61efd5a1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:42.612225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:42.633257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4f3c-7584-4dc1-8d9e-38f72cd7c916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:42.634390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4f3c-7584-4dc1-8d9e-38f72cd7c916 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:42.634773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4f3c-7584-4dc1-8d9e-38f72cd7c916 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:42.635150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4f3c-7584-4dc1-8d9e-38f72cd7c916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:42.636169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4f3c-7584-4dc1-8d9e-38f72cd7c916 None admin] RBAC: Adding query filter params (role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374, user.id=cc27c4ba7fc8495b91efdaf391ca4359) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:42.636712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4f3c-7584-4dc1-8d9e-38f72cd7c916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:42.641394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4f3c-7584-4dc1-8d9e-38f72cd7c916 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:42.645112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 106/222] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:42 2022] GET /identity/v3/role_assignments?user.id=cc27c4ba7fc8495b91efdaf391ca4359&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:42.645637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a2c2a-5f37-40bb-a9e6-5e63d70f48e0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:42.720677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94502d-7a93-4263-a6c8-24b530e1888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:42.721346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94502d-7a93-4263-a6c8-24b530e1888d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:42.721620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94502d-7a93-4263-a6c8-24b530e1888d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:42.721888 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94502d-7a93-4263-a6c8-24b530e1888d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:42.734696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe60aa-8353-4492-98cb-08e1b2fe77ae None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:42.735760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe60aa-8353-4492-98cb-08e1b2fe77ae None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:42.736267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe60aa-8353-4492-98cb-08e1b2fe77ae None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:42.738249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 107/223] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:42.743351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94502d-7a93-4263-a6c8-24b530e1888d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:42.744696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da421-b26a-49e3-82bc-b71f0848d1e1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:42.744985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da421-b26a-49e3-82bc-b71f0848d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:42.745262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da421-b26a-49e3-82bc-b71f0848d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3c94502d-7a93-4263-a6c8-24b530e1888d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:42.748572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:42.749470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:42.750279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:42.751504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 117/224] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:42.756445 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94502d-7a93-4263-a6c8-24b530e1888d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:42.788381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d54da421-b26a-49e3-82bc-b71f0848d1e1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:42.789615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d54da421-b26a-49e3-82bc-b71f0848d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:42.822441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:42.931728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 108/225] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:42.933869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94502d-7a93-4263-a6c8-24b530e1888d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:43.013073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:43.013620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:43.013959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:43.014235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:43.014916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:43.015221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:43.016735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:43.031120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 118/226] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:43.037489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da421-b26a-49e3-82bc-b71f0848d1e1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:43.182640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da421-b26a-49e3-82bc-b71f0848d1e1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:43.192635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:43.229356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8db00-3220-49a1-b87e-9c3170034376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:43.230006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8db00-3220-49a1-b87e-9c3170034376 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:43.230324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8db00-3220-49a1-b87e-9c3170034376 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:43.230599 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8db00-3220-49a1-b87e-9c3170034376 None admin] PATH_INFO: `/v3/users/6fe1e399f3be4a1495299fae0f0ca715` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:43.231770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8db00-3220-49a1-b87e-9c3170034376 None admin] RBAC: Authorizing `identity:get_user(user_id=6fe1e399f3be4a1495299fae0f0ca715)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:43.233233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8db00-3220-49a1-b87e-9c3170034376 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:43.235273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 109/227] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:43 2022] GET /identity/v3/users/6fe1e399f3be4a1495299fae0f0ca715 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:43.241149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:43.309154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8db00-3220-49a1-b87e-9c3170034376 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:43.412680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:43.428326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0c1dc-4ba3-4913-b0ae-200143c89bf9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:43.478001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8db00-3220-49a1-b87e-9c3170034376 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:43.496041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6264f-8f0b-4d92-94fc-6fc0738503e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:43.496684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6264f-8f0b-4d92-94fc-6fc0738503e9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:43.496962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6264f-8f0b-4d92-94fc-6fc0738503e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:43.497238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6264f-8f0b-4d92-94fc-6fc0738503e9 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:43.500185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6264f-8f0b-4d92-94fc-6fc0738503e9 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:43.502090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6264f-8f0b-4d92-94fc-6fc0738503e9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:43.506614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 119/228] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:43 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:43.509807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6264f-8f0b-4d92-94fc-6fc0738503e9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:43.557331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a735d-b350-4267-9bc2-6eacc2fa1779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:43.558350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a735d-b350-4267-9bc2-6eacc2fa1779 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:43.558824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a735d-b350-4267-9bc2-6eacc2fa1779 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:43.559445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a735d-b350-4267-9bc2-6eacc2fa1779 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:43.588248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a735d-b350-4267-9bc2-6eacc2fa1779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1c7a735d-b350-4267-9bc2-6eacc2fa1779 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:43.593768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:43.594889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:43.595719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:43.598131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 110/229] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:04:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:04:43.608401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a735d-b350-4267-9bc2-6eacc2fa1779 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:43.647173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:43.685837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6264f-8f0b-4d92-94fc-6fc0738503e9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:43.747981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359344e2-ebfd-4c1b-a342-546d123cf432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:43.749218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359344e2-ebfd-4c1b-a342-546d123cf432 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:43.750728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359344e2-ebfd-4c1b-a342-546d123cf432 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:43.751452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359344e2-ebfd-4c1b-a342-546d123cf432 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/users/6fe1e399f3be4a1495299fae0f0ca715/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:43.755967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359344e2-ebfd-4c1b-a342-546d123cf432 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb8bbbfbab8e46d49858d9ef99a96374, user_id=6fe1e399f3be4a1495299fae0f0ca715, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:43.757841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359344e2-ebfd-4c1b-a342-546d123cf432 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:43.774722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a735d-b350-4267-9bc2-6eacc2fa1779 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:43.779777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 120/230] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:43 2022] PUT /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/users/6fe1e399f3be4a1495299fae0f0ca715/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:43.825396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:43.826335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:43.826838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:43.827259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:43.828117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:43.828584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:43.830323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:43.845254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 111/231] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:04:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:43.851862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359344e2-ebfd-4c1b-a342-546d123cf432 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:43.897238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:43.995615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359344e2-ebfd-4c1b-a342-546d123cf432 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:44.044471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e7b9-038a-47c3-a1ea-288fea1e47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:44.045157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e7b9-038a-47c3-a1ea-288fea1e47ff None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.045440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e7b9-038a-47c3-a1ea-288fea1e47ff None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.045710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e7b9-038a-47c3-a1ea-288fea1e47ff None admin] PATH_INFO: `/v3/users/6fe1e399f3be4a1495299fae0f0ca715` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.046853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45e7b9-038a-47c3-a1ea-288fea1e47ff None admin] RBAC: Authorizing `identity:get_user(user_id=6fe1e399f3be4a1495299fae0f0ca715)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:44.048519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45e7b9-038a-47c3-a1ea-288fea1e47ff None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:44.051593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 121/232] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:43 2022] GET /identity/v3/users/6fe1e399f3be4a1495299fae0f0ca715 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:44.058631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:44.192002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:44.196519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52527a94-296b-404d-a0ea-8f430c477cf0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:44.234868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c27945-80f6-4dbd-82d3-a563e018660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:44.235713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c27945-80f6-4dbd-82d3-a563e018660b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.235989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c27945-80f6-4dbd-82d3-a563e018660b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.236262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c27945-80f6-4dbd-82d3-a563e018660b None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.237427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c27945-80f6-4dbd-82d3-a563e018660b None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:44.238955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c27945-80f6-4dbd-82d3-a563e018660b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:44.241134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 112/233] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:44 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:44.246701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e7b9-038a-47c3-a1ea-288fea1e47ff None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:44.362636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:44.379163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e7b9-038a-47c3-a1ea-288fea1e47ff None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:44.421053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e5969-1b43-4913-9314-cbd64ba382dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:44.421732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e5969-1b43-4913-9314-cbd64ba382dd None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.422010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e5969-1b43-4913-9314-cbd64ba382dd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.422280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e5969-1b43-4913-9314-cbd64ba382dd None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/users/6fe1e399f3be4a1495299fae0f0ca715/roles/0f4eb127111b4fdabec35268db5bde9e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.424588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e5969-1b43-4913-9314-cbd64ba382dd None admin] RBAC: Authorizing `identity:create_grant(project_id=bb8bbbfbab8e46d49858d9ef99a96374, user_id=6fe1e399f3be4a1495299fae0f0ca715, role_id=0f4eb127111b4fdabec35268db5bde9e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:44.426130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e5969-1b43-4913-9314-cbd64ba382dd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:44.443688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 122/234] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:44 2022] PUT /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/users/6fe1e399f3be4a1495299fae0f0ca715/roles/0f4eb127111b4fdabec35268db5bde9e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:44.493809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13dc0e-cfaa-49c2-88b4-09f1b0358ced None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.494143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13dc0e-cfaa-49c2-88b4-09f1b0358ced None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.494358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13dc0e-cfaa-49c2-88b4-09f1b0358ced None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.496076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 113/235] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:44.502713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40575069-906d-4cf5-a52e-44987dd9ed43 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.503020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40575069-906d-4cf5-a52e-44987dd9ed43 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.503316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40575069-906d-4cf5-a52e-44987dd9ed43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.528955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-40575069-906d-4cf5-a52e-44987dd9ed43 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:44.530269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-40575069-906d-4cf5-a52e-44987dd9ed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:44.617751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 123/236] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:44.657058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6bc43-a8cb-4052-bb2d-49dd2bea5ed6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.657437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6bc43-a8cb-4052-bb2d-49dd2bea5ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.657597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6bc43-a8cb-4052-bb2d-49dd2bea5ed6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.659028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 114/237] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:44.664660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dc7b-65f8-4424-b362-b7fff881fc8b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.664948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dc7b-65f8-4424-b362-b7fff881fc8b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.665210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dc7b-65f8-4424-b362-b7fff881fc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.688396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ef14dc7b-65f8-4424-b362-b7fff881fc8b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:44.689632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ef14dc7b-65f8-4424-b362-b7fff881fc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:44.764180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 124/238] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:44.798837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140def3e-183c-4fa6-a768-53da8cbcaf95 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.798837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140def3e-183c-4fa6-a768-53da8cbcaf95 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.798837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140def3e-183c-4fa6-a768-53da8cbcaf95 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.799958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 115/239] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:44.805754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423c2ae-0f8e-4761-a2a4-5f8dfc797d34 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:44.806380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423c2ae-0f8e-4761-a2a4-5f8dfc797d34 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:44.806837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423c2ae-0f8e-4761-a2a4-5f8dfc797d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:44.825797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a423c2ae-0f8e-4761-a2a4-5f8dfc797d34 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:44.827339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a423c2ae-0f8e-4761-a2a4-5f8dfc797d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:44.830712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:44.901429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 125/240] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:44.990360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140def3e-183c-4fa6-a768-53da8cbcaf95 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.124042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140def3e-183c-4fa6-a768-53da8cbcaf95 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:45.131408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423c2ae-0f8e-4761-a2a4-5f8dfc797d34 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.174760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d0793-58ec-4865-91f9-174600236555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:45.175730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0793-58ec-4865-91f9-174600236555 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:45.176009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0793-58ec-4865-91f9-174600236555 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:45.177149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0793-58ec-4865-91f9-174600236555 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:45.190114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d0793-58ec-4865-91f9-174600236555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c85d0793-58ec-4865-91f9-174600236555 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:45.197511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:45.198403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:45.199197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:45.199824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 116/241] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:45.210518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d0793-58ec-4865-91f9-174600236555 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.255105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423c2ae-0f8e-4761-a2a4-5f8dfc797d34 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:45.282095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:45.302025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3175e-01c7-4b66-b8ce-0b3d9720087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:45.303245 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3175e-01c7-4b66-b8ce-0b3d9720087a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:45.303663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3175e-01c7-4b66-b8ce-0b3d9720087a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:45.303941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3175e-01c7-4b66-b8ce-0b3d9720087a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:45.305780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3175e-01c7-4b66-b8ce-0b3d9720087a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:45.307546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3175e-01c7-4b66-b8ce-0b3d9720087a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:45.309751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 126/242] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:45.311175 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3175e-01c7-4b66-b8ce-0b3d9720087a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.324792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:45.354013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d0793-58ec-4865-91f9-174600236555 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:45.404712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:45.405250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:45.405528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:45.405801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:45.407200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:45.407535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:45.409656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:45.422314 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 117/243] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:45.423918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.435154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3175e-01c7-4b66-b8ce-0b3d9720087a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:45.487336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784107-fb2b-4f9e-a18c-39a17dce6316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:45.488089 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784107-fb2b-4f9e-a18c-39a17dce6316 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:45.488459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784107-fb2b-4f9e-a18c-39a17dce6316 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:45.488796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784107-fb2b-4f9e-a18c-39a17dce6316 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:45.503970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da784107-fb2b-4f9e-a18c-39a17dce6316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-da784107-fb2b-4f9e-a18c-39a17dce6316 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:45.509396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:45.510265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:45.511048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 127/244] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:45.511810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784107-fb2b-4f9e-a18c-39a17dce6316 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.534921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:45.550692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fa229-e1b6-4903-ad61-67ba79ccc372 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:45.588032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:45.589338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef988c8-fbf9-4ad4-96b6-0aca578ac0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:45.590170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef988c8-fbf9-4ad4-96b6-0aca578ac0d4 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:45.590589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef988c8-fbf9-4ad4-96b6-0aca578ac0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:45.591706 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef988c8-fbf9-4ad4-96b6-0aca578ac0d4 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:45.591984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef988c8-fbf9-4ad4-96b6-0aca578ac0d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:45.593749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef988c8-fbf9-4ad4-96b6-0aca578ac0d4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:45.622048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 118/245] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:04:45 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:45.623674 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef988c8-fbf9-4ad4-96b6-0aca578ac0d4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.629678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784107-fb2b-4f9e-a18c-39a17dce6316 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:45.689680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89baa-893a-48b4-b1ba-20da193916f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:45.690312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89baa-893a-48b4-b1ba-20da193916f4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:45.691347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89baa-893a-48b4-b1ba-20da193916f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:45.691698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89baa-893a-48b4-b1ba-20da193916f4 None admin] PATH_INFO: `/v3/users/ffc5880840f74b928fad758e77939e85` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:45.710223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89baa-893a-48b4-b1ba-20da193916f4 None admin] RBAC: Authorizing `identity:get_user(user_id=ffc5880840f74b928fad758e77939e85)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:45.711834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89baa-893a-48b4-b1ba-20da193916f4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:45.714131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 128/246] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:45 2022] GET /identity/v3/users/ffc5880840f74b928fad758e77939e85 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:45.721626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89baa-893a-48b4-b1ba-20da193916f4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.855493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef988c8-fbf9-4ad4-96b6-0aca578ac0d4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:45.898487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89baa-893a-48b4-b1ba-20da193916f4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:45.920197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:45.921167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:45.921634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:45.922109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:45.923051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:45.923847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:45.925837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:45.936052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 119/247] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:45.946523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:45.950524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:45.961650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c96de7-672e-4bd1-8d52-350eb33dfa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:45.962100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96de7-672e-4bd1-8d52-350eb33dfa16 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:45.962380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96de7-672e-4bd1-8d52-350eb33dfa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:45.962652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96de7-672e-4bd1-8d52-350eb33dfa16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:45.971605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:46.058488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c96de7-672e-4bd1-8d52-350eb33dfa16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:46.060371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c96de7-672e-4bd1-8d52-350eb33dfa16 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:46.109785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff513f2-ba7b-4030-978a-acb4b36e549a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:46.155457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c203e2-5f61-44d2-b06e-d38bad8f6538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:46.156124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c203e2-5f61-44d2-b06e-d38bad8f6538 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:46.156406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c203e2-5f61-44d2-b06e-d38bad8f6538 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:46.156723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c203e2-5f61-44d2-b06e-d38bad8f6538 None admin] PATH_INFO: `/v3/users/cc27c4ba7fc8495b91efdaf391ca4359` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:46.158236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c203e2-5f61-44d2-b06e-d38bad8f6538 None admin] RBAC: Authorizing `identity:get_user(user_id=cc27c4ba7fc8495b91efdaf391ca4359)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:46.159920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c203e2-5f61-44d2-b06e-d38bad8f6538 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:46.162038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 120/248] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:45 2022] GET /identity/v3/users/cc27c4ba7fc8495b91efdaf391ca4359 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:46.167349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c203e2-5f61-44d2-b06e-d38bad8f6538 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:46.173772 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 129/249] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:46.180188 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c96de7-672e-4bd1-8d52-350eb33dfa16 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:46.226293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:46.273824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:46.327531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c96de7-672e-4bd1-8d52-350eb33dfa16 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:46.358183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c203e2-5f61-44d2-b06e-d38bad8f6538 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:46.378027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31ded2-ee80-43b0-b5e1-618b60b3bd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:46.378684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31ded2-ee80-43b0-b5e1-618b60b3bd9d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:46.378960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31ded2-ee80-43b0-b5e1-618b60b3bd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:46.379256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31ded2-ee80-43b0-b5e1-618b60b3bd9d None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:46.380428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31ded2-ee80-43b0-b5e1-618b60b3bd9d None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:46.381920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31ded2-ee80-43b0-b5e1-618b60b3bd9d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:46.383975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 130/250] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:46 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:46.390488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31ded2-ee80-43b0-b5e1-618b60b3bd9d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:46.438658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2357fb-b572-4e0d-a246-f1ca931b08e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:46.439349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2357fb-b572-4e0d-a246-f1ca931b08e0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:46.439629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2357fb-b572-4e0d-a246-f1ca931b08e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:46.439899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2357fb-b572-4e0d-a246-f1ca931b08e0 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:46.441958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2357fb-b572-4e0d-a246-f1ca931b08e0 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:46.443475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2357fb-b572-4e0d-a246-f1ca931b08e0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:46.445182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 121/251] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:46 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:46.451703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2357fb-b572-4e0d-a246-f1ca931b08e0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:46.570641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:46.579357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31ded2-ee80-43b0-b5e1-618b60b3bd9d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:46.619997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2357fb-b572-4e0d-a246-f1ca931b08e0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:46.665629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54159f75-8379-4417-b72f-bb97d41cbc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:46.666463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54159f75-8379-4417-b72f-bb97d41cbc47 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:46.666840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54159f75-8379-4417-b72f-bb97d41cbc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:46.667210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54159f75-8379-4417-b72f-bb97d41cbc47 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/users/cc27c4ba7fc8495b91efdaf391ca4359/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:46.669402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54159f75-8379-4417-b72f-bb97d41cbc47 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb8bbbfbab8e46d49858d9ef99a96374, user_id=cc27c4ba7fc8495b91efdaf391ca4359, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:46.670980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54159f75-8379-4417-b72f-bb97d41cbc47 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:46.674314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5f253-a1c6-40f4-ba87-13ebd291a023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:46.675053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5f253-a1c6-40f4-ba87-13ebd291a023 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:46.675380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5f253-a1c6-40f4-ba87-13ebd291a023 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:46.675655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5f253-a1c6-40f4-ba87-13ebd291a023 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:46.676397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5f253-a1c6-40f4-ba87-13ebd291a023 None admin] RBAC: Adding query filter params (role.id=5681843b3fe64a0ebe56433cfca360e7, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374, user.id=ffc5880840f74b928fad758e77939e85) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:46.676706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5f253-a1c6-40f4-ba87-13ebd291a023 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:46.678296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5f253-a1c6-40f4-ba87-13ebd291a023 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:46.684851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 131/252] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:46 2022] GET /identity/v3/role_assignments?user.id=ffc5880840f74b928fad758e77939e85&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=5681843b3fe64a0ebe56433cfca360e7 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:46.685946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6d7cc-a0b8-41b3-8176-0adec311cd70 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:46.686226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6d7cc-a0b8-41b3-8176-0adec311cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:46.686496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6d7cc-a0b8-41b3-8176-0adec311cd70 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:46.696185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 132/253] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:46.699376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 122/254] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:46 2022] PUT /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/users/cc27c4ba7fc8495b91efdaf391ca4359/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:46.701887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70eee1-9da3-427a-89ab-01f3a9c4381a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:46.702158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70eee1-9da3-427a-89ab-01f3a9c4381a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:46.703990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70eee1-9da3-427a-89ab-01f3a9c4381a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:46.736410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ae70eee1-9da3-427a-89ab-01f3a9c4381a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:46.737652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ae70eee1-9da3-427a-89ab-01f3a9c4381a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:46.842111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 133/255] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:47.023970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6048a36-f438-4844-b1c2-e098251b7b21 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:47.024917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6048a36-f438-4844-b1c2-e098251b7b21 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:47.025391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6048a36-f438-4844-b1c2-e098251b7b21 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:47.026999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 123/256] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:47.032636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60956cb2-e9bb-4add-a55b-12dc019a7632 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:47.032918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60956cb2-e9bb-4add-a55b-12dc019a7632 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:47.033187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60956cb2-e9bb-4add-a55b-12dc019a7632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:47.057196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-60956cb2-e9bb-4add-a55b-12dc019a7632 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:47.058375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-60956cb2-e9bb-4add-a55b-12dc019a7632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:47.149894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 134/257] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:47.187388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6048a36-f438-4844-b1c2-e098251b7b21 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:47.190886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:47.339537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6048a36-f438-4844-b1c2-e098251b7b21 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:47.389076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd013e-592f-429e-baec-ba8cdf4f1c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:47.389721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd013e-592f-429e-baec-ba8cdf4f1c56 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:47.390000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd013e-592f-429e-baec-ba8cdf4f1c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:47.390269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd013e-592f-429e-baec-ba8cdf4f1c56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:47.410466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd013e-592f-429e-baec-ba8cdf4f1c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c0fd013e-592f-429e-baec-ba8cdf4f1c56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:47.415906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:47.416839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:47.418069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 124/258] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:04:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:47.425101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60956cb2-e9bb-4add-a55b-12dc019a7632 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:47.454290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:47.485895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd013e-592f-429e-baec-ba8cdf4f1c56 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:47.546994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60956cb2-e9bb-4add-a55b-12dc019a7632 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:47.587135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:47.588022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:47.588434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:47.588838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:47.589663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:47.590096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:47.591662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:47.601338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 135/259] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:47.601700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:47.611570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:47.655269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd013e-592f-429e-baec-ba8cdf4f1c56 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:47.691369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39cff96-e078-4150-8e38-7a349d589747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:47.691369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cff96-e078-4150-8e38-7a349d589747 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:47.691369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cff96-e078-4150-8e38-7a349d589747 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:47.691369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cff96-e078-4150-8e38-7a349d589747 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:47.700887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39cff96-e078-4150-8e38-7a349d589747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e39cff96-e078-4150-8e38-7a349d589747 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:47.706378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:47.711780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:47.712575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 125/260] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:04:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:04:47.713919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39cff96-e078-4150-8e38-7a349d589747 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:47.759596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:47.799627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ea5f-fca1-45a1-ae0d-a185e7d6398f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:47.868001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a874-074a-491c-9ff0-1baa285a22bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:47.868649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a874-074a-491c-9ff0-1baa285a22bf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:47.868927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a874-074a-491c-9ff0-1baa285a22bf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:47.869230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a874-074a-491c-9ff0-1baa285a22bf None admin] PATH_INFO: `/v3/users/6fe1e399f3be4a1495299fae0f0ca715` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:47.873514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10a874-074a-491c-9ff0-1baa285a22bf None admin] RBAC: Authorizing `identity:get_user(user_id=6fe1e399f3be4a1495299fae0f0ca715)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:47.873514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10a874-074a-491c-9ff0-1baa285a22bf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:47.879328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 136/261] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:47 2022] GET /identity/v3/users/6fe1e399f3be4a1495299fae0f0ca715 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:47.882676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a874-074a-491c-9ff0-1baa285a22bf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:47.900025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39cff96-e078-4150-8e38-7a349d589747 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:47.938639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:47.962472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:47.963003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:47.963300 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:47.963576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:47.964264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:47.964569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:47.965983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:47.986449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 126/262] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:04:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:47.994199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:48.071126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a874-074a-491c-9ff0-1baa285a22bf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:48.152026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a16c6b-3e39-425d-a537-a134de141a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:48.152672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a16c6b-3e39-425d-a537-a134de141a7f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:48.152955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a16c6b-3e39-425d-a537-a134de141a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:48.153232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a16c6b-3e39-425d-a537-a134de141a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:48.186855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:48.196771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76322cac-8abe-474d-8910-23bae5e5c213 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:48.250678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a16c6b-3e39-425d-a537-a134de141a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:48.256962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a16c6b-3e39-425d-a537-a134de141a7f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:48.276729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd48dd-a6f5-45f9-a920-62ad003b6bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:48.277336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd48dd-a6f5-45f9-a920-62ad003b6bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:48.277672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd48dd-a6f5-45f9-a920-62ad003b6bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:48.277950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd48dd-a6f5-45f9-a920-62ad003b6bd4 None admin] PATH_INFO: `/v3/users/6fe1e399f3be4a1495299fae0f0ca715` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:48.280958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd48dd-a6f5-45f9-a920-62ad003b6bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=6fe1e399f3be4a1495299fae0f0ca715)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:48.282749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd48dd-a6f5-45f9-a920-62ad003b6bd4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:48.285173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 127/263] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:47 2022] GET /identity/v3/users/6fe1e399f3be4a1495299fae0f0ca715 => generated 283 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:48.290444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd48dd-a6f5-45f9-a920-62ad003b6bd4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:48.409585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 137/264] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:04:48.415642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:48.416665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a16c6b-3e39-425d-a537-a134de141a7f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:48.503154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:48.505897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd48dd-a6f5-45f9-a920-62ad003b6bd4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:48.556125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382d441-c333-4319-a97d-dcfe941bb037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:48.556843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d441-c333-4319-a97d-dcfe941bb037 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:48.557192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d441-c333-4319-a97d-dcfe941bb037 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:48.557529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d441-c333-4319-a97d-dcfe941bb037 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:48.572924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a16c6b-3e39-425d-a537-a134de141a7f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:48.612704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d88d82-14be-4dd0-a61b-71c37125a082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:48.613594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d88d82-14be-4dd0-a61b-71c37125a082 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:48.613933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d88d82-14be-4dd0-a61b-71c37125a082 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:48.614181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d88d82-14be-4dd0-a61b-71c37125a082 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:48.616511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d88d82-14be-4dd0-a61b-71c37125a082 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:48.620705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d88d82-14be-4dd0-a61b-71c37125a082 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:48.627279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 138/265] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:48 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:48.636375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d88d82-14be-4dd0-a61b-71c37125a082 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:48.666301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382d441-c333-4319-a97d-dcfe941bb037 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:48.668502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382d441-c333-4319-a97d-dcfe941bb037 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:48.743226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:48.808716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 128/266] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:04:48.814742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382d441-c333-4319-a97d-dcfe941bb037 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:48.889662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d88d82-14be-4dd0-a61b-71c37125a082 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:48.911916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:48.931055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e79a6-675f-4de1-89a9-8f6a40d1e393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:48.931837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e79a6-675f-4de1-89a9-8f6a40d1e393 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:48.932208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e79a6-675f-4de1-89a9-8f6a40d1e393 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:48.932572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e79a6-675f-4de1-89a9-8f6a40d1e393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:48.933387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e79a6-675f-4de1-89a9-8f6a40d1e393 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374, user.id=6fe1e399f3be4a1495299fae0f0ca715) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:48.933885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e79a6-675f-4de1-89a9-8f6a40d1e393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:48.935438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e79a6-675f-4de1-89a9-8f6a40d1e393 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:48.942142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 139/267] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:48 2022] GET /identity/v3/role_assignments?user.id=6fe1e399f3be4a1495299fae0f0ca715&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:48.943265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24200-3978-4743-b798-463c44422ab6 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:48.943623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24200-3978-4743-b798-463c44422ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:48.944111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24200-3978-4743-b798-463c44422ab6 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:48.945736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 140/268] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:48.946817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48c50-5016-449c-a78d-73eaaae9126e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:48.947153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48c50-5016-449c-a78d-73eaaae9126e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:48.947504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48c50-5016-449c-a78d-73eaaae9126e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:48.948968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 141/269] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:48.954643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db530841-764a-4b5e-89b8-248c42bb524e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:48.955120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db530841-764a-4b5e-89b8-248c42bb524e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:48.955477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db530841-764a-4b5e-89b8-248c42bb524e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:48.972043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382d441-c333-4319-a97d-dcfe941bb037 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:48.991992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-db530841-764a-4b5e-89b8-248c42bb524e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:48.995381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-db530841-764a-4b5e-89b8-248c42bb524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:49.060496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c014e37-8e44-4cc2-b260-de036cd38ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:49.061063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c014e37-8e44-4cc2-b260-de036cd38ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.061354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c014e37-8e44-4cc2-b260-de036cd38ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.061624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c014e37-8e44-4cc2-b260-de036cd38ed6 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.063399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c014e37-8e44-4cc2-b260-de036cd38ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:49.065497 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c014e37-8e44-4cc2-b260-de036cd38ed6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:49.067756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 129/270] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:48 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:49.075928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78838a-d838-4637-a258-15cc1a94c011 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.076241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78838a-d838-4637-a258-15cc1a94c011 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.076530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78838a-d838-4637-a258-15cc1a94c011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.122512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8d78838a-d838-4637-a258-15cc1a94c011 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:49.123757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8d78838a-d838-4637-a258-15cc1a94c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:49.143478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 142/271] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:49.146044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db530841-764a-4b5e-89b8-248c42bb524e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:49.169008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:49.259498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 130/272] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:49.300059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db530841-764a-4b5e-89b8-248c42bb524e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:49.311614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:49.335691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d681cc6-bd7d-4479-8938-8c7ed5ebccae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:49.336309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681cc6-bd7d-4479-8938-8c7ed5ebccae None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.336584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681cc6-bd7d-4479-8938-8c7ed5ebccae None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.336848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681cc6-bd7d-4479-8938-8c7ed5ebccae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.337637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d681cc6-bd7d-4479-8938-8c7ed5ebccae None admin] RBAC: Adding query filter params (role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374, user.id=6fe1e399f3be4a1495299fae0f0ca715) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:49.338172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d681cc6-bd7d-4479-8938-8c7ed5ebccae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:49.339711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d681cc6-bd7d-4479-8938-8c7ed5ebccae None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:49.346830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 143/273] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:49 2022] GET /identity/v3/role_assignments?user.id=6fe1e399f3be4a1495299fae0f0ca715&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:49.608209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24f876-d86f-4f36-8024-98a32b81d409 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.608554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24f876-d86f-4f36-8024-98a32b81d409 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.608654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24f876-d86f-4f36-8024-98a32b81d409 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.610683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 131/274] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:49.616580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01d83-1ecf-457d-ad13-12b41d7a739a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.617142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01d83-1ecf-457d-ad13-12b41d7a739a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.617618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01d83-1ecf-457d-ad13-12b41d7a739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.637940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-aeb01d83-1ecf-457d-ad13-12b41d7a739a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:49.639470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aeb01d83-1ecf-457d-ad13-12b41d7a739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:49.646912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24f876-d86f-4f36-8024-98a32b81d409 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:49.727554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 144/275] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:49.729830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb01d83-1ecf-457d-ad13-12b41d7a739a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:49.779757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24f876-d86f-4f36-8024-98a32b81d409 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:49.825232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d7651-559f-4878-be16-d881c10cbd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:49.826038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7651-559f-4878-be16-d881c10cbd5b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.826370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7651-559f-4878-be16-d881c10cbd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.826647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7651-559f-4878-be16-d881c10cbd5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.827857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d7651-559f-4878-be16-d881c10cbd5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:49.829856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d7651-559f-4878-be16-d881c10cbd5b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:49.835282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 132/276] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:49.835282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f4d0-57ca-49c4-b65e-51209d9a8ea0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.835282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f4d0-57ca-49c4-b65e-51209d9a8ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.835282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f4d0-57ca-49c4-b65e-51209d9a8ea0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.835282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 133/277] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:49.839520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37f4d0-57ca-49c4-b65e-51209d9a8ea0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:49.852200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb01d83-1ecf-457d-ad13-12b41d7a739a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:49.854914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:49.864063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:49.896544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb0af1-770a-4dee-85bc-4d9ae225804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:49.897185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb0af1-770a-4dee-85bc-4d9ae225804d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.897444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb0af1-770a-4dee-85bc-4d9ae225804d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.897722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb0af1-770a-4dee-85bc-4d9ae225804d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.908015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb0af1-770a-4dee-85bc-4d9ae225804d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d3cb0af1-770a-4dee-85bc-4d9ae225804d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:49.912907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:49.913868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:49.914658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 145/278] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:49.916149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6deda-c15d-414f-b30b-a8ea45fc2339 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:49.916149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6deda-c15d-414f-b30b-a8ea45fc2339 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:49.916302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6deda-c15d-414f-b30b-a8ea45fc2339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:49.934164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0bb6deda-c15d-414f-b30b-a8ea45fc2339 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:49.935389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0bb6deda-c15d-414f-b30b-a8ea45fc2339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:49.961789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37f4d0-57ca-49c4-b65e-51209d9a8ea0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.000337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d32cb-5d3c-485c-ad57-56b62b640232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.001480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d32cb-5d3c-485c-ad57-56b62b640232 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.001765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d32cb-5d3c-485c-ad57-56b62b640232 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.002027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d32cb-5d3c-485c-ad57-56b62b640232 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.002694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d32cb-5d3c-485c-ad57-56b62b640232 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:50.004210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d32cb-5d3c-485c-ad57-56b62b640232 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:50.006275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 146/279] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:50.016968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6deda-c15d-414f-b30b-a8ea45fc2339 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.036905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 134/280] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:04:49 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:50.036905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d32cb-5d3c-485c-ad57-56b62b640232 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.131991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:50.160574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:50.170106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d32cb-5d3c-485c-ad57-56b62b640232 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.174383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6deda-c15d-414f-b30b-a8ea45fc2339 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.219807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def0bfe-8bf2-4de1-9c9b-c007e99e6ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.220447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def0bfe-8bf2-4de1-9c9b-c007e99e6ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.220822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def0bfe-8bf2-4de1-9c9b-c007e99e6ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.221192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def0bfe-8bf2-4de1-9c9b-c007e99e6ae3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.222175 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.222831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.223065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.223429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.224096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:50.224712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:50.226103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:50.233448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def0bfe-8bf2-4de1-9c9b-c007e99e6ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:50.235590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 147/281] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2def0bfe-8bf2-4de1-9c9b-c007e99e6ae3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:50.240228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:50.241513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:50.242490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 135/282] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:50.244872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.249440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def0bfe-8bf2-4de1-9c9b-c007e99e6ae3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.378437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def0bfe-8bf2-4de1-9c9b-c007e99e6ae3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.382406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103332f3-56a7-4a13-90ee-9f42a90192fd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.422156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.422774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.423054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.423358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.424027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:50.424325 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:50.425606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d95b16-ff81-416d-8adf-42418abd241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.425726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:50.426516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d95b16-ff81-416d-8adf-42418abd241a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.427157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:50.428055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d95b16-ff81-416d-8adf-42418abd241a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.428565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d95b16-ff81-416d-8adf-42418abd241a None admin] PATH_INFO: `/v3/users/028d5d84322847d796150b761ac54c43` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.437324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 136/283] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:50.439036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.456750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d95b16-ff81-416d-8adf-42418abd241a None admin] RBAC: Authorizing `identity:get_user(user_id=028d5d84322847d796150b761ac54c43)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:50.458692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d95b16-ff81-416d-8adf-42418abd241a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:50.461476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 148/284] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/users/028d5d84322847d796150b761ac54c43 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:50.462354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d95b16-ff81-416d-8adf-42418abd241a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.485138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:50.582179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d95b16-ff81-416d-8adf-42418abd241a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.582450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9ac0-3f7a-4501-a0aa-7b31bcc8c53e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.620240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e237b-2304-4861-a007-0ba8eac0b22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.621225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e237b-2304-4861-a007-0ba8eac0b22c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.621731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e237b-2304-4861-a007-0ba8eac0b22c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.622195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e237b-2304-4861-a007-0ba8eac0b22c None admin] PATH_INFO: `/v3/users/ffc5880840f74b928fad758e77939e85` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.623584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e237b-2304-4861-a007-0ba8eac0b22c None admin] RBAC: Authorizing `identity:get_user(user_id=ffc5880840f74b928fad758e77939e85)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:50.625328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e237b-2304-4861-a007-0ba8eac0b22c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:50.626238 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ad6dc-6b1d-4c47-9cde-00d8eb05b385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.626876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad6dc-6b1d-4c47-9cde-00d8eb05b385 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.627154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad6dc-6b1d-4c47-9cde-00d8eb05b385 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.627454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad6dc-6b1d-4c47-9cde-00d8eb05b385 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.628124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 149/285] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/users/ffc5880840f74b928fad758e77939e85 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:50.629096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e237b-2304-4861-a007-0ba8eac0b22c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.645304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ad6dc-6b1d-4c47-9cde-00d8eb05b385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-de7ad6dc-6b1d-4c47-9cde-00d8eb05b385 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:50.652202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:50.653103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:50.653904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:50.656669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 137/286] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:50.657842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ad6dc-6b1d-4c47-9cde-00d8eb05b385 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.741566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:50.744781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:50.764248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e237b-2304-4861-a007-0ba8eac0b22c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.786350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ad6dc-6b1d-4c47-9cde-00d8eb05b385 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:50.807553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.808682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.808972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.809296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.810041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] RBAC: Adding query filter params (role.id=07097b9e79ae4ff1bf60ddef2b681c51, user.id=028d5d84322847d796150b761ac54c43, scope.system=all) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:50.810503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:50.812467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:50.819888 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 150/287] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/role_assignments?user.id=028d5d84322847d796150b761ac54c43&scope.system=all&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:50.824248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.833295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef49ba-3880-4e0d-aebe-7a68a8617424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:50.834360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef49ba-3880-4e0d-aebe-7a68a8617424 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:50.834837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef49ba-3880-4e0d-aebe-7a68a8617424 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:50.835513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef49ba-3880-4e0d-aebe-7a68a8617424 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:50.837985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef49ba-3880-4e0d-aebe-7a68a8617424 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:50.840870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef49ba-3880-4e0d-aebe-7a68a8617424 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:50.845660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 138/288] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:50.859359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef49ba-3880-4e0d-aebe-7a68a8617424 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:50.962656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9ac9-d871-4146-9e7e-cc163c749e38 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:51.003908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:51.004528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:51.004808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:51.005078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:51.005926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:51.006500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:51.007920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:51.018494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 151/289] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:51.024540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:51.043339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:51.106595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef49ba-3880-4e0d-aebe-7a68a8617424 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:51.144815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:51.155280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780041a-59bb-43e0-a554-094ee2a55d4d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:51.184010 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c31540-24ca-448a-81f1-88a895db242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:51.185063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c31540-24ca-448a-81f1-88a895db242e None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:51.185597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c31540-24ca-448a-81f1-88a895db242e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:51.186113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c31540-24ca-448a-81f1-88a895db242e None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/users/ffc5880840f74b928fad758e77939e85/roles/5681843b3fe64a0ebe56433cfca360e7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:51.189562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2be9ba-ab91-4d17-a3aa-8039eeff48d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:51.190295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2be9ba-ab91-4d17-a3aa-8039eeff48d3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:51.190643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2be9ba-ab91-4d17-a3aa-8039eeff48d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:51.190960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c31540-24ca-448a-81f1-88a895db242e None admin] RBAC: Authorizing `identity:create_grant(project_id=bb8bbbfbab8e46d49858d9ef99a96374, user_id=ffc5880840f74b928fad758e77939e85, role_id=5681843b3fe64a0ebe56433cfca360e7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:51.191083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2be9ba-ab91-4d17-a3aa-8039eeff48d3 None admin] PATH_INFO: `/v3/users/cc27c4ba7fc8495b91efdaf391ca4359` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:51.192249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2be9ba-ab91-4d17-a3aa-8039eeff48d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cc27c4ba7fc8495b91efdaf391ca4359)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:51.192989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c31540-24ca-448a-81f1-88a895db242e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:51.193861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2be9ba-ab91-4d17-a3aa-8039eeff48d3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:51.195734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 152/290] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:51 2022] GET /identity/v3/users/cc27c4ba7fc8495b91efdaf391ca4359 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:51.202116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2be9ba-ab91-4d17-a3aa-8039eeff48d3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:51.217797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 139/291] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:04:50 2022] PUT /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/users/ffc5880840f74b928fad758e77939e85/roles/5681843b3fe64a0ebe56433cfca360e7 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:04:51.325841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:51.358671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2be9ba-ab91-4d17-a3aa-8039eeff48d3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:51.409028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b13a42-0038-44b2-97c2-08d19a088b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:51.410058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b13a42-0038-44b2-97c2-08d19a088b2a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:51.410543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b13a42-0038-44b2-97c2-08d19a088b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:51.411002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b13a42-0038-44b2-97c2-08d19a088b2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:51.505205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b13a42-0038-44b2-97c2-08d19a088b2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:51.507052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b13a42-0038-44b2-97c2-08d19a088b2a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:51.598046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:51.628618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 153/292] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:04:51.633997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c31540-24ca-448a-81f1-88a895db242e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:51.788484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c31540-24ca-448a-81f1-88a895db242e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:51.831737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8583d-c99f-4913-9794-249c73eb7aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:51.832389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8583d-c99f-4913-9794-249c73eb7aad None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:51.832666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8583d-c99f-4913-9794-249c73eb7aad None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:51.832938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8583d-c99f-4913-9794-249c73eb7aad None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:51.834088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8583d-c99f-4913-9794-249c73eb7aad None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:51.835570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8583d-c99f-4913-9794-249c73eb7aad None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:51.837595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 140/293] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:51 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:51.843226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b13a42-0038-44b2-97c2-08d19a088b2a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:51.995435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b13a42-0038-44b2-97c2-08d19a088b2a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:52.049538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cc33e-1caa-4dc4-90c0-1e01a79f2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:52.050083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cc33e-1caa-4dc4-90c0-1e01a79f2a6b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:52.050366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cc33e-1caa-4dc4-90c0-1e01a79f2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:52.050666 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cc33e-1caa-4dc4-90c0-1e01a79f2a6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:52.051438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cc33e-1caa-4dc4-90c0-1e01a79f2a6b None admin] RBAC: Adding query filter params (role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374, user.id=cc27c4ba7fc8495b91efdaf391ca4359) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:52.051743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cc33e-1caa-4dc4-90c0-1e01a79f2a6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:52.053254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cc33e-1caa-4dc4-90c0-1e01a79f2a6b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:52.064221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 154/294] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:51 2022] GET /identity/v3/role_assignments?user.id=cc27c4ba7fc8495b91efdaf391ca4359&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:52.477462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1c474-39cd-4410-9775-62c89a7722e4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:52.478158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1c474-39cd-4410-9775-62c89a7722e4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:52.478639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1c474-39cd-4410-9775-62c89a7722e4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:52.480552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 141/295] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:52.486098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afccd8-d845-4612-a9fe-1d2fea09f0c3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:52.486378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afccd8-d845-4612-a9fe-1d2fea09f0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:52.486649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afccd8-d845-4612-a9fe-1d2fea09f0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:52.509863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-55afccd8-d845-4612-a9fe-1d2fea09f0c3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:52.511054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-55afccd8-d845-4612-a9fe-1d2fea09f0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:52.547623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d314e5-dd42-4e0f-a115-8ce32061da9c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:52.548239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d314e5-dd42-4e0f-a115-8ce32061da9c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:52.548708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d314e5-dd42-4e0f-a115-8ce32061da9c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:52.550577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 142/296] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:52.557821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308aa65-d641-4978-87c6-71674d1a0228 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:52.558447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308aa65-d641-4978-87c6-71674d1a0228 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:52.558774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:52.559127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308aa65-d641-4978-87c6-71674d1a0228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:52.580861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b308aa65-d641-4978-87c6-71674d1a0228 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:52.582686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b308aa65-d641-4978-87c6-71674d1a0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:52.614315 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 155/297] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:52.648992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:52.673644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 143/298] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:52.893192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afccd8-d845-4612-a9fe-1d2fea09f0c3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:52.957527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308aa65-d641-4978-87c6-71674d1a0228 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:53.010905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afccd8-d845-4612-a9fe-1d2fea09f0c3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:53.049123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ce128-7fb7-4687-8b68-924ef0e3e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:53.049922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce128-7fb7-4687-8b68-924ef0e3e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.050197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce128-7fb7-4687-8b68-924ef0e3e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.050471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce128-7fb7-4687-8b68-924ef0e3e6c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.067488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ce128-7fb7-4687-8b68-924ef0e3e6c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-799ce128-7fb7-4687-8b68-924ef0e3e6c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:53.072672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:53.073573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:53.074372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 156/299] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:53.085515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ce128-7fb7-4687-8b68-924ef0e3e6c7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:53.088708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308aa65-d641-4978-87c6-71674d1a0228 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:53.129494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:53.132213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eec464-a74c-4075-a823-277961241e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:53.133060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eec464-a74c-4075-a823-277961241e36 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.133509 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eec464-a74c-4075-a823-277961241e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.133775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eec464-a74c-4075-a823-277961241e36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.135038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eec464-a74c-4075-a823-277961241e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:53.136586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eec464-a74c-4075-a823-277961241e36 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:53.138415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 144/300] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:53.143423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ce9e-baaa-470b-aef7-c872538a52a0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.143423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ce9e-baaa-470b-aef7-c872538a52a0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.143423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ce9e-baaa-470b-aef7-c872538a52a0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.143423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 145/301] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:53.146337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602ce9e-baaa-470b-aef7-c872538a52a0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:53.204354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:53.225501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ce128-7fb7-4687-8b68-924ef0e3e6c7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:53.264142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad814b-dc40-45f9-bbef-f8f54dcf8248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:53.264759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad814b-dc40-45f9-bbef-f8f54dcf8248 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.265304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad814b-dc40-45f9-bbef-f8f54dcf8248 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.265466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad814b-dc40-45f9-bbef-f8f54dcf8248 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.266231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad814b-dc40-45f9-bbef-f8f54dcf8248 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:53.266548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad814b-dc40-45f9-bbef-f8f54dcf8248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:53.268044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad814b-dc40-45f9-bbef-f8f54dcf8248 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:53.274351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602ce9e-baaa-470b-aef7-c872538a52a0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:53.277251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 157/302] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:53.282139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73309f5-1a0f-4271-8e56-eff21f3684d5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.282139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73309f5-1a0f-4271-8e56-eff21f3684d5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.282139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73309f5-1a0f-4271-8e56-eff21f3684d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.298735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d73309f5-1a0f-4271-8e56-eff21f3684d5 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:53.300082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d73309f5-1a0f-4271-8e56-eff21f3684d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:53.321523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf203c-c425-4e79-b352-4751e1423665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:53.322149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf203c-c425-4e79-b352-4751e1423665 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.322425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf203c-c425-4e79-b352-4751e1423665 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.322695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf203c-c425-4e79-b352-4751e1423665 None admin] PATH_INFO: `/v3/groups` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.324322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf203c-c425-4e79-b352-4751e1423665 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:53.327580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf203c-c425-4e79-b352-4751e1423665 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:53.363260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 146/303] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Aug 31 12:04:53 2022] POST /identity/v3/groups => generated 229 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:53.363260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf203c-c425-4e79-b352-4751e1423665 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:53.399528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 158/304] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:53.501163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf203c-c425-4e79-b352-4751e1423665 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:53.514762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:53.541481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c14d82-9059-4a1b-8ff5-0fc786384a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:53.542123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c14d82-9059-4a1b-8ff5-0fc786384a20 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.542396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c14d82-9059-4a1b-8ff5-0fc786384a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.542665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c14d82-9059-4a1b-8ff5-0fc786384a20 None admin] PATH_INFO: `/v3/users/356ecba04f03492d97c5b5a093ff8781` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.559705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c14d82-9059-4a1b-8ff5-0fc786384a20 None admin] RBAC: Authorizing `identity:get_user(user_id=356ecba04f03492d97c5b5a093ff8781)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:53.561278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c14d82-9059-4a1b-8ff5-0fc786384a20 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:53.563273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 147/305] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:53 2022] GET /identity/v3/users/356ecba04f03492d97c5b5a093ff8781 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:53.569617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73309f5-1a0f-4271-8e56-eff21f3684d5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:53.602334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:53.673578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c14d82-9059-4a1b-8ff5-0fc786384a20 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:53.693010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73309f5-1a0f-4271-8e56-eff21f3684d5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:53.742090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22341d87-b1c8-478a-9ce6-cfecdbb9b05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:53.742750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22341d87-b1c8-478a-9ce6-cfecdbb9b05d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.743019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22341d87-b1c8-478a-9ce6-cfecdbb9b05d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.743325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22341d87-b1c8-478a-9ce6-cfecdbb9b05d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.744050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22341d87-b1c8-478a-9ce6-cfecdbb9b05d None admin] RBAC: Adding query filter params (role.id=5681843b3fe64a0ebe56433cfca360e7, user.id=356ecba04f03492d97c5b5a093ff8781, scope.system=all) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:53.744352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22341d87-b1c8-478a-9ce6-cfecdbb9b05d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:53.745813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22341d87-b1c8-478a-9ce6-cfecdbb9b05d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:53.752369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 159/306] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 31 12:04:53 2022] GET /identity/v3/role_assignments?user.id=356ecba04f03492d97c5b5a093ff8781&scope.system=all&role.id=5681843b3fe64a0ebe56433cfca360e7 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:53.761162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a740a74-a35b-4f6a-8b28-bf8830de6bf1 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.761475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a740a74-a35b-4f6a-8b28-bf8830de6bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.762677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a740a74-a35b-4f6a-8b28-bf8830de6bf1 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.763074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 160/307] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:53.768975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6fe42-4ddf-405c-8cc4-66da78fa611b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.769293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6fe42-4ddf-405c-8cc4-66da78fa611b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.769514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6fe42-4ddf-405c-8cc4-66da78fa611b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.798045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9ba6fe42-4ddf-405c-8cc4-66da78fa611b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:53.799331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9ba6fe42-4ddf-405c-8cc4-66da78fa611b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:53.815363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c14d82-9059-4a1b-8ff5-0fc786384a20 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:53.866697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28748c92-c0ab-4c21-b998-280753a0dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:53.867638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28748c92-c0ab-4c21-b998-280753a0dc6e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:53.867910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28748c92-c0ab-4c21-b998-280753a0dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:53.868605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28748c92-c0ab-4c21-b998-280753a0dc6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:53.884596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28748c92-c0ab-4c21-b998-280753a0dc6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:53.888856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 161/308] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-28748c92-c0ab-4c21-b998-280753a0dc6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:53.894692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:53.895735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:53.896710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:53.903641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 148/309] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:04:53.905811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6fe42-4ddf-405c-8cc4-66da78fa611b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:53.939724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:54.031351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6fe42-4ddf-405c-8cc4-66da78fa611b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:54.067877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:54.068490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:54.068760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:54.069025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:54.069692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:54.069987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:54.071390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:54.080732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 162/310] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:54.086242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28748c92-c0ab-4c21-b998-280753a0dc6e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:54.126840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:54.171912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:54.198653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28748c92-c0ab-4c21-b998-280753a0dc6e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:54.242634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9a83-7f14-4356-b921-6bd9515882b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:54.243978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9a83-7f14-4356-b921-6bd9515882b8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:54.244462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9a83-7f14-4356-b921-6bd9515882b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:54.245051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9a83-7f14-4356-b921-6bd9515882b8 None admin] PATH_INFO: `/v3/users/028d5d84322847d796150b761ac54c43` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:54.246700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9a83-7f14-4356-b921-6bd9515882b8 None admin] RBAC: Authorizing `identity:get_user(user_id=028d5d84322847d796150b761ac54c43)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:54.248589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9a83-7f14-4356-b921-6bd9515882b8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:54.250458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 149/311] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:54 2022] GET /identity/v3/users/028d5d84322847d796150b761ac54c43 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:54.256248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9a83-7f14-4356-b921-6bd9515882b8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:54.286481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:54.318130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1468b3f-3a5d-410a-b807-b53509ed40b7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:54.357625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6db44c-451e-47e5-a19d-8cd9041bbb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:54.358416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6db44c-451e-47e5-a19d-8cd9041bbb63 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:54.358833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6db44c-451e-47e5-a19d-8cd9041bbb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:54.359193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6db44c-451e-47e5-a19d-8cd9041bbb63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:54.366958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:54.373657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9a83-7f14-4356-b921-6bd9515882b8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:54.375767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6db44c-451e-47e5-a19d-8cd9041bbb63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bf6db44c-451e-47e5-a19d-8cd9041bbb63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:54.381303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:54.382279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:54.383059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 163/312] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:54.389233 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6db44c-451e-47e5-a19d-8cd9041bbb63 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:54.426956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5a693-fd05-44b3-9520-2dfff24aa1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:54.428185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5a693-fd05-44b3-9520-2dfff24aa1b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:54.428717 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5a693-fd05-44b3-9520-2dfff24aa1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:54.429242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5a693-fd05-44b3-9520-2dfff24aa1b6 None admin] PATH_INFO: `/v3/system/users/028d5d84322847d796150b761ac54c43/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:54.431765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5a693-fd05-44b3-9520-2dfff24aa1b6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=028d5d84322847d796150b761ac54c43, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:54.434410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5a693-fd05-44b3-9520-2dfff24aa1b6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:54.449369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 150/313] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Aug 31 12:04:54 2022] PUT /identity/v3/system/users/028d5d84322847d796150b761ac54c43/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:04:54.514833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6db44c-451e-47e5-a19d-8cd9041bbb63 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:54.556275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:54.556959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:54.557239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:54.557548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:54.558276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:54.558577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:54.560234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:54.567382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:54.577757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 164/314] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:54.580254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5a693-fd05-44b3-9520-2dfff24aa1b6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:54.691767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5a693-fd05-44b3-9520-2dfff24aa1b6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:54.727833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f109eaf-ea9d-4b75-ac50-80d932f3da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:54.728477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109eaf-ea9d-4b75-ac50-80d932f3da2b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:54.728756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109eaf-ea9d-4b75-ac50-80d932f3da2b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:54.729027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109eaf-ea9d-4b75-ac50-80d932f3da2b None admin] PATH_INFO: `/v3/users/ffc5880840f74b928fad758e77939e85` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:54.730178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f109eaf-ea9d-4b75-ac50-80d932f3da2b None admin] RBAC: Authorizing `identity:get_user(user_id=ffc5880840f74b928fad758e77939e85)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:54.731683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f109eaf-ea9d-4b75-ac50-80d932f3da2b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:54.733515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 151/315] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:54 2022] GET /identity/v3/users/ffc5880840f74b928fad758e77939e85 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:54.739127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:54.848592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca801d-86cf-461a-aa1b-0dca8c574edc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:54.886068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd67ad-4cfb-4011-b4e8-45522fb916b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:54.886986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd67ad-4cfb-4011-b4e8-45522fb916b0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:54.887457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd67ad-4cfb-4011-b4e8-45522fb916b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:54.887891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd67ad-4cfb-4011-b4e8-45522fb916b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:54.967096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd67ad-4cfb-4011-b4e8-45522fb916b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:54.968840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd67ad-4cfb-4011-b4e8-45522fb916b0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:54.971509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:55.080141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 165/316] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:04:55.085638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f109eaf-ea9d-4b75-ac50-80d932f3da2b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:55.118873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:55.193022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f109eaf-ea9d-4b75-ac50-80d932f3da2b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:55.229318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d0aec-98fb-4468-a517-71bd4312c16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:55.229961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0aec-98fb-4468-a517-71bd4312c16d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:55.230232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0aec-98fb-4468-a517-71bd4312c16d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:55.230496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0aec-98fb-4468-a517-71bd4312c16d None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:55.231669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d0aec-98fb-4468-a517-71bd4312c16d None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:55.233195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d0aec-98fb-4468-a517-71bd4312c16d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:55.235486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 152/317] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:04:55 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:55.241391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd67ad-4cfb-4011-b4e8-45522fb916b0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:55.350353 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:55.354167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd67ad-4cfb-4011-b4e8-45522fb916b0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:55.390789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c8703-a382-44b5-89a0-d598ba5d4c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:55.391849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c8703-a382-44b5-89a0-d598ba5d4c81 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:55.392321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c8703-a382-44b5-89a0-d598ba5d4c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:55.392780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c8703-a382-44b5-89a0-d598ba5d4c81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:55.393731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c8703-a382-44b5-89a0-d598ba5d4c81 None admin] RBAC: Adding query filter params (role.id=5681843b3fe64a0ebe56433cfca360e7, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374, user.id=ffc5880840f74b928fad758e77939e85) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:55.394238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c8703-a382-44b5-89a0-d598ba5d4c81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:55.395953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c8703-a382-44b5-89a0-d598ba5d4c81 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:55.402855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 166/318] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:04:55 2022] GET /identity/v3/role_assignments?user.id=ffc5880840f74b928fad758e77939e85&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=5681843b3fe64a0ebe56433cfca360e7 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:55.503853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a4d80-5f43-450a-a8c1-32dc6592aa6b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:55.504604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a4d80-5f43-450a-a8c1-32dc6592aa6b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:55.505074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a4d80-5f43-450a-a8c1-32dc6592aa6b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:55.507114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 153/319] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:55.513057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11cfe-a128-471c-ae12-f9ff5a054e06 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:55.513670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11cfe-a128-471c-ae12-f9ff5a054e06 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:55.514137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11cfe-a128-471c-ae12-f9ff5a054e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:55.535015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-06c11cfe-a128-471c-ae12-f9ff5a054e06 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:55.536646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-06c11cfe-a128-471c-ae12-f9ff5a054e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:55.612631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 167/320] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:55.799066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaac145-4170-429f-b34b-71a8ac5ce990 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:55.799599 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaac145-4170-429f-b34b-71a8ac5ce990 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:55.799709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaac145-4170-429f-b34b-71a8ac5ce990 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:55.800985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 154/321] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:55.806647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62237518-cb85-473f-bddc-6d8718b97e77 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:55.806939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62237518-cb85-473f-bddc-6d8718b97e77 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:55.807213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62237518-cb85-473f-bddc-6d8718b97e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:55.830113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-62237518-cb85-473f-bddc-6d8718b97e77 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:55.831332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-62237518-cb85-473f-bddc-6d8718b97e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:55.880310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaac145-4170-429f-b34b-71a8ac5ce990 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:55.918862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 168/322] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:55.997297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:56.005716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaac145-4170-429f-b34b-71a8ac5ce990 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:56.045105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20ab50-74d1-4b8d-9587-ff9b65601b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:56.046116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20ab50-74d1-4b8d-9587-ff9b65601b05 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:56.046591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20ab50-74d1-4b8d-9587-ff9b65601b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:56.047430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20ab50-74d1-4b8d-9587-ff9b65601b05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:56.048848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20ab50-74d1-4b8d-9587-ff9b65601b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:56.050472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20ab50-74d1-4b8d-9587-ff9b65601b05 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:56.052666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 155/323] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:04:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:56.057907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62237518-cb85-473f-bddc-6d8718b97e77 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:56.085536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:56.168564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62237518-cb85-473f-bddc-6d8718b97e77 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:56.188513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20ab50-74d1-4b8d-9587-ff9b65601b05 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:56.222557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cba01-c2dc-4a9f-b9a6-19dce827f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:56.223539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cba01-c2dc-4a9f-b9a6-19dce827f9fc None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:56.223980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cba01-c2dc-4a9f-b9a6-19dce827f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:56.224727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cba01-c2dc-4a9f-b9a6-19dce827f9fc None admin] PATH_INFO: `/v3/groups` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:56.225760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046cba01-c2dc-4a9f-b9a6-19dce827f9fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:56.227960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046cba01-c2dc-4a9f-b9a6-19dce827f9fc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:56.251470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 169/324] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Aug 31 12:04:56 2022] POST /identity/v3/groups => generated 226 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:04:56.331676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20ab50-74d1-4b8d-9587-ff9b65601b05 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:56.374181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559ca9f-977d-4a9d-a1a8-3592c1b4c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:56.375325 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559ca9f-977d-4a9d-a1a8-3592c1b4c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:56.375789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559ca9f-977d-4a9d-a1a8-3592c1b4c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:56.376324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559ca9f-977d-4a9d-a1a8-3592c1b4c2d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:56.389109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559ca9f-977d-4a9d-a1a8-3592c1b4c2d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a559ca9f-977d-4a9d-a1a8-3592c1b4c2d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:56.394828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:56.395859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:04:56.396723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:56.398657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 156/325] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:56.404055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cba01-c2dc-4a9f-b9a6-19dce827f9fc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:56.479882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91cb28-8bc6-4795-9589-abf41c7cf117 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:56.480108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91cb28-8bc6-4795-9589-abf41c7cf117 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:56.480267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91cb28-8bc6-4795-9589-abf41c7cf117 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:56.481907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 157/326] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:04:56.489212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5006a2c-d8df-4f9b-996a-726272c2c0ab None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:56.489513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5006a2c-d8df-4f9b-996a-726272c2c0ab None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:56.489755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5006a2c-d8df-4f9b-996a-726272c2c0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:56.500140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:56.514122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e5006a2c-d8df-4f9b-996a-726272c2c0ab None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:56.515985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e5006a2c-d8df-4f9b-996a-726272c2c0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:56.518169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cba01-c2dc-4a9f-b9a6-19dce827f9fc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:56.542976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:56.556488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:56.557101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:56.557374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:56.557643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:56.558332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:56.558603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:56.560229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:56.573198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 170/327] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:56.579105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:56.615370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 158/328] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:56.716755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffee5a-336e-4c64-9cea-31fee2eaf67b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:56.760424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2e2f9-7c8e-4317-ab3b-339dabb77194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:56.760992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2e2f9-7c8e-4317-ab3b-339dabb77194 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:56.761266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2e2f9-7c8e-4317-ab3b-339dabb77194 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:56.761533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2e2f9-7c8e-4317-ab3b-339dabb77194 None admin] PATH_INFO: `/v3/users/356ecba04f03492d97c5b5a093ff8781` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:56.762678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2e2f9-7c8e-4317-ab3b-339dabb77194 None admin] RBAC: Authorizing `identity:get_user(user_id=356ecba04f03492d97c5b5a093ff8781)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:56.764169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2e2f9-7c8e-4317-ab3b-339dabb77194 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:56.766907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 171/329] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:56 2022] GET /identity/v3/users/356ecba04f03492d97c5b5a093ff8781 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:04:56.772647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5006a2c-d8df-4f9b-996a-726272c2c0ab None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:56.893852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5006a2c-d8df-4f9b-996a-726272c2c0ab None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:56.914714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2e2f9-7c8e-4317-ab3b-339dabb77194 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:56.936855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205acc2-d6cf-4661-944e-2b2a73393877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:56.938079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205acc2-d6cf-4661-944e-2b2a73393877 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:56.938412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205acc2-d6cf-4661-944e-2b2a73393877 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:56.938692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205acc2-d6cf-4661-944e-2b2a73393877 None admin] PATH_INFO: `/v3/system/users/356ecba04f03492d97c5b5a093ff8781/roles/5681843b3fe64a0ebe56433cfca360e7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:56.940326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205acc2-d6cf-4661-944e-2b2a73393877 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=356ecba04f03492d97c5b5a093ff8781, role_id=5681843b3fe64a0ebe56433cfca360e7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:56.941830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:56.943264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205acc2-d6cf-4661-944e-2b2a73393877 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:56.957080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 159/330] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Aug 31 12:04:56 2022] PUT /identity/v3/system/users/356ecba04f03492d97c5b5a093ff8781/roles/5681843b3fe64a0ebe56433cfca360e7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:04:57.068365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2e2f9-7c8e-4317-ab3b-339dabb77194 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:57.118323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327ab81-7f9a-4808-8b3d-639c25ae28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:57.119120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ab81-7f9a-4808-8b3d-639c25ae28a6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:57.119351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ab81-7f9a-4808-8b3d-639c25ae28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:57.119626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ab81-7f9a-4808-8b3d-639c25ae28a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:57.136174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327ab81-7f9a-4808-8b3d-639c25ae28a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5327ab81-7f9a-4808-8b3d-639c25ae28a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:57.142356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:57.143332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:57.144847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 172/331] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:57.150536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205acc2-d6cf-4661-944e-2b2a73393877 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:57.176069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:57.269039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205acc2-d6cf-4661-944e-2b2a73393877 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:57.307745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:57.308365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:57.308643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:57.308917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:57.309588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:57.309887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:57.311297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:57.320924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 160/332] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:57.326416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327ab81-7f9a-4808-8b3d-639c25ae28a6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:57.418351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:57.445637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327ab81-7f9a-4808-8b3d-639c25ae28a6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:57.482536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cadff-4794-4164-9c8d-8d77b9809abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:57.483255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cadff-4794-4164-9c8d-8d77b9809abb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:57.483545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cadff-4794-4164-9c8d-8d77b9809abb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:57.483820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cadff-4794-4164-9c8d-8d77b9809abb None admin] PATH_INFO: `/v3/users/028d5d84322847d796150b761ac54c43` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:57.485083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cadff-4794-4164-9c8d-8d77b9809abb None admin] RBAC: Authorizing `identity:get_user(user_id=028d5d84322847d796150b761ac54c43)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:57.486556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cadff-4794-4164-9c8d-8d77b9809abb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:57.488621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 173/333] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:04:57 2022] GET /identity/v3/users/028d5d84322847d796150b761ac54c43 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:57.494715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:57.589227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:57.617639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384241d-8b19-409b-9bea-dcd0c4d6e9b3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:57.663119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eec54-cb40-452d-99db-71cba722828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:57.663767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eec54-cb40-452d-99db-71cba722828f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:57.664053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eec54-cb40-452d-99db-71cba722828f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:57.664326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eec54-cb40-452d-99db-71cba722828f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:57.665058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5eec54-cb40-452d-99db-71cba722828f None admin] RBAC: Adding query filter params (role.id=07097b9e79ae4ff1bf60ddef2b681c51, user.id=028d5d84322847d796150b761ac54c43, scope.system=all) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:04:57.665437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5eec54-cb40-452d-99db-71cba722828f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:04:57.666913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5eec54-cb40-452d-99db-71cba722828f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:04:57.680233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 161/334] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 31 12:04:57 2022] GET /identity/v3/role_assignments?user.id=028d5d84322847d796150b761ac54c43&scope.system=all&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:04:58.853996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a2dc7-e4c6-4d75-80e4-6b4b1986f250 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:58.854706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a2dc7-e4c6-4d75-80e4-6b4b1986f250 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:58.854706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a2dc7-e4c6-4d75-80e4-6b4b1986f250 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:58.856158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 174/335] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:58.865365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652fffc-3a79-4ef2-84c0-2dceea636642 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:58.866255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652fffc-3a79-4ef2-84c0-2dceea636642 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:58.866677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652fffc-3a79-4ef2-84c0-2dceea636642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:58.888420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3652fffc-3a79-4ef2-84c0-2dceea636642 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:58.889930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3652fffc-3a79-4ef2-84c0-2dceea636642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:58.957063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4575d-8035-4bc1-9626-6db888e9bcad None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:58.957660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4575d-8035-4bc1-9626-6db888e9bcad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:58.958113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4575d-8035-4bc1-9626-6db888e9bcad None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:58.963331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 175/336] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:58.970380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342f489-425f-47a9-a539-40f6d38c313e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:58.970980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342f489-425f-47a9-a539-40f6d38c313e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:58.971454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342f489-425f-47a9-a539-40f6d38c313e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:58.981983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 162/337] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:04:59.008314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4342f489-425f-47a9-a539-40f6d38c313e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:59.009815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4342f489-425f-47a9-a539-40f6d38c313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:59.101945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 176/338] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:59.515950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652fffc-3a79-4ef2-84c0-2dceea636642 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:59.573589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342f489-425f-47a9-a539-40f6d38c313e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:59.600973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:59.661952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652fffc-3a79-4ef2-84c0-2dceea636642 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:59.673869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:04:59.709280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342f489-425f-47a9-a539-40f6d38c313e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:59.717524 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d0553-b1e9-4174-9003-af982598f85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:59.718503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d0553-b1e9-4174-9003-af982598f85d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:59.718930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d0553-b1e9-4174-9003-af982598f85d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:59.719388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d0553-b1e9-4174-9003-af982598f85d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:59.741059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d0553-b1e9-4174-9003-af982598f85d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4d7d0553-b1e9-4174-9003-af982598f85d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:59.748793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:59.749802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:04:59.751067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 163/339] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:04:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:04:59.757947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8614b7a-3f0b-4f40-8054-4aa472621fcf None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:59.758256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8614b7a-3f0b-4f40-8054-4aa472621fcf None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:59.758496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8614b7a-3f0b-4f40-8054-4aa472621fcf None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:59.761605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94771be4-12b4-41e3-8396-cc0800817057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:04:59.762261 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771be4-12b4-41e3-8396-cc0800817057 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:59.762536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771be4-12b4-41e3-8396-cc0800817057 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:59.762805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771be4-12b4-41e3-8396-cc0800817057 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:59.763510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 164/340] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:04:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:04:59.766499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8614b7a-3f0b-4f40-8054-4aa472621fcf None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:04:59.784861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94771be4-12b4-41e3-8396-cc0800817057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-94771be4-12b4-41e3-8396-cc0800817057 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:04:59.790129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:04:59.792612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 177/341] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:04:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:04:59.794642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02cfd-436d-488a-a222-8119cfb782d4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:04:59.796266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02cfd-436d-488a-a222-8119cfb782d4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:04:59.796266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02cfd-436d-488a-a222-8119cfb782d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:04:59.826142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f1f02cfd-436d-488a-a222-8119cfb782d4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:04:59.827397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f1f02cfd-436d-488a-a222-8119cfb782d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:04:59.977468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8614b7a-3f0b-4f40-8054-4aa472621fcf None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:04:59.979353 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 178/342] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:04:59.980299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f02cfd-436d-488a-a222-8119cfb782d4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:00.040688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:00.057448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:00.058069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.058376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.058704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.059498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:00.059801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:00.061259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:00.075974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:00.079475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 165/343] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:04:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:00.093110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:00.129895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f02cfd-436d-488a-a222-8119cfb782d4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:00.166116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:00.166747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.167023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.167326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.168010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:00.168313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:00.169711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:00.178924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 179/344] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:04:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:00.184641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:00.282140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388e525-0022-406a-a66d-e2bddd0f86a6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:00.305775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076e8b5-73d7-465b-b036-e0f63f2d37f4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:00.330203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:00.341369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b5679-c01a-4b1c-998a-246a31629651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:00.341981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5679-c01a-4b1c-998a-246a31629651 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.342262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5679-c01a-4b1c-998a-246a31629651 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.342534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5679-c01a-4b1c-998a-246a31629651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.356900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fa047-91bd-428e-aebe-41d71e5e4711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:00.357515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fa047-91bd-428e-aebe-41d71e5e4711 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.357784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fa047-91bd-428e-aebe-41d71e5e4711 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.358048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fa047-91bd-428e-aebe-41d71e5e4711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.425571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b5679-c01a-4b1c-998a-246a31629651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:00.427138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b5679-c01a-4b1c-998a-246a31629651 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:00.459384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:00.475268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fa047-91bd-428e-aebe-41d71e5e4711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:00.477066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fa047-91bd-428e-aebe-41d71e5e4711 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:00.552107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 180/345] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:05:00.554069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115911-210b-4fb3-a4e0-1cafd6795e9e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.554350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115911-210b-4fb3-a4e0-1cafd6795e9e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.554620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115911-210b-4fb3-a4e0-1cafd6795e9e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.556564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 181/346] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:00.557678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0115911-210b-4fb3-a4e0-1cafd6795e9e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:00.589158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:00.611778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 166/347] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:05:00.612738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fa047-91bd-428e-aebe-41d71e5e4711 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:00.686588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0115911-210b-4fb3-a4e0-1cafd6795e9e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:00.727613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:00.729229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f001e83-6f5d-4721-be2f-38689c631c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:00.730080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001e83-6f5d-4721-be2f-38689c631c06 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.730481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001e83-6f5d-4721-be2f-38689c631c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.730753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001e83-6f5d-4721-be2f-38689c631c06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.748912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fa047-91bd-428e-aebe-41d71e5e4711 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:00.751579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f001e83-6f5d-4721-be2f-38689c631c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-2f001e83-6f5d-4721-be2f-38689c631c06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:00.756711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:00.757706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:00.758908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 182/348] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:00.759840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d514f-500f-4f21-b63a-91c3e7430eb9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.759840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d514f-500f-4f21-b63a-91c3e7430eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.759840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d514f-500f-4f21-b63a-91c3e7430eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.784218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7b7d514f-500f-4f21-b63a-91c3e7430eb9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:00.785535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7b7d514f-500f-4f21-b63a-91c3e7430eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:00.808879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076786-d628-466c-a23c-365e38932fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:00.809458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076786-d628-466c-a23c-365e38932fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.809733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076786-d628-466c-a23c-365e38932fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.810002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076786-d628-466c-a23c-365e38932fb7 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.811202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d076786-d628-466c-a23c-365e38932fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:00.812852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d076786-d628-466c-a23c-365e38932fb7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:00.819318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 167/349] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:00 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:00.819318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076786-d628-466c-a23c-365e38932fb7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:00.887698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 183/350] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:00.889055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec30d9-634a-4526-ac32-11db00b42b9c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:00.889408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec30d9-634a-4526-ac32-11db00b42b9c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:00.889797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec30d9-634a-4526-ac32-11db00b42b9c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:00.892363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 184/351] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:00.892819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec30d9-634a-4526-ac32-11db00b42b9c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:00.911921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:00.963120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076786-d628-466c-a23c-365e38932fb7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.003040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb4c29-52ac-4881-bf32-c1ecc21814d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.004851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb4c29-52ac-4881-bf32-c1ecc21814d9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.005571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb4c29-52ac-4881-bf32-c1ecc21814d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.006124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb4c29-52ac-4881-bf32-c1ecc21814d9 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.008245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb4c29-52ac-4881-bf32-c1ecc21814d9 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:01.010397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb4c29-52ac-4881-bf32-c1ecc21814d9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:01.013249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 168/352] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:00 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:01.014342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb4c29-52ac-4881-bf32-c1ecc21814d9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.017934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec30d9-634a-4526-ac32-11db00b42b9c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.048315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:01.060292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4536-d58b-488e-9519-3f94f7e11b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.060879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4536-d58b-488e-9519-3f94f7e11b0a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.061199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4536-d58b-488e-9519-3f94f7e11b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.061473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4536-d58b-488e-9519-3f94f7e11b0a None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.062200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee4536-d58b-488e-9519-3f94f7e11b0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:01.062498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee4536-d58b-488e-9519-3f94f7e11b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:01.065170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee4536-d58b-488e-9519-3f94f7e11b0a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:01.075845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 185/353] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:05:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:01.081822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a52c0-c4b1-417f-9646-7dcc428b2c76 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.082147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a52c0-c4b1-417f-9646-7dcc428b2c76 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.082300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a52c0-c4b1-417f-9646-7dcc428b2c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.104964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a30a52c0-c4b1-417f-9646-7dcc428b2c76 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:01.106341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a30a52c0-c4b1-417f-9646-7dcc428b2c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:01.134371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb4c29-52ac-4881-bf32-c1ecc21814d9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.173534 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7336f5-7cc3-4514-a625-c6074d38a614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.174632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7336f5-7cc3-4514-a625-c6074d38a614 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.175098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7336f5-7cc3-4514-a625-c6074d38a614 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.175636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7336f5-7cc3-4514-a625-c6074d38a614 None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.183358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7336f5-7cc3-4514-a625-c6074d38a614 None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:01.185293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7336f5-7cc3-4514-a625-c6074d38a614 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:01.188008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 169/354] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:01.189370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7336f5-7cc3-4514-a625-c6074d38a614 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.196468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 186/355] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:01.197699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a52c0-c4b1-417f-9646-7dcc428b2c76 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.207918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:01.303091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:01.315291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7336f5-7cc3-4514-a625-c6074d38a614 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.317256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a52c0-c4b1-417f-9646-7dcc428b2c76 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.356832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55706e-1364-46f4-a981-489817808a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.357474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55706e-1364-46f4-a981-489817808a66 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.357751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55706e-1364-46f4-a981-489817808a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.358022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55706e-1364-46f4-a981-489817808a66 None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.359929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55706e-1364-46f4-a981-489817808a66 None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:01.362130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55706e-1364-46f4-a981-489817808a66 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:01.364152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 170/356] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:01.365257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55706e-1364-46f4-a981-489817808a66 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.367439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ab437-2f0c-4aee-b204-9c091cc8121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.368154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab437-2f0c-4aee-b204-9c091cc8121f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.368606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab437-2f0c-4aee-b204-9c091cc8121f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.368839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab437-2f0c-4aee-b204-9c091cc8121f None admin] PATH_INFO: `/v3/users/356ecba04f03492d97c5b5a093ff8781` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.370994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ab437-2f0c-4aee-b204-9c091cc8121f None admin] RBAC: Authorizing `identity:get_user(user_id=356ecba04f03492d97c5b5a093ff8781)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:01.372860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ab437-2f0c-4aee-b204-9c091cc8121f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:01.374691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 187/357] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/users/356ecba04f03492d97c5b5a093ff8781 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:01.375324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ab437-2f0c-4aee-b204-9c091cc8121f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.466739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:01.479343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55706e-1364-46f4-a981-489817808a66 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.500446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ab437-2f0c-4aee-b204-9c091cc8121f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.519656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2166b3-e5eb-4d8d-8126-b2f3cc323394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.520271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2166b3-e5eb-4d8d-8126-b2f3cc323394 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.520552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2166b3-e5eb-4d8d-8126-b2f3cc323394 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.520823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2166b3-e5eb-4d8d-8126-b2f3cc323394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.521540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2166b3-e5eb-4d8d-8126-b2f3cc323394 None admin] RBAC: Adding query filter params (group.id=289105f07754455c9316a1ee7452c1e4, role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=de5700cfe30946bd8e0082d5b4964f98) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:01.521838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2166b3-e5eb-4d8d-8126-b2f3cc323394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:01.523320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2166b3-e5eb-4d8d-8126-b2f3cc323394 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:01.529264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 171/358] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/role_assignments?group.id=289105f07754455c9316a1ee7452c1e4&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:01.530333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66d145-5c33-4126-bb83-747c6435c866 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.530683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66d145-5c33-4126-bb83-747c6435c866 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.531174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66d145-5c33-4126-bb83-747c6435c866 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.532755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 172/359] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:01.533387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66d145-5c33-4126-bb83-747c6435c866 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.546654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba26c73-98f9-443e-b903-09d0e4ae1f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.547498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba26c73-98f9-443e-b903-09d0e4ae1f85 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.547718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba26c73-98f9-443e-b903-09d0e4ae1f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.547985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba26c73-98f9-443e-b903-09d0e4ae1f85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.560878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba26c73-98f9-443e-b903-09d0e4ae1f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-eba26c73-98f9-443e-b903-09d0e4ae1f85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:01.566107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:01.567067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:01.567889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 188/360] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:01.568546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba26c73-98f9-443e-b903-09d0e4ae1f85 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.575657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:01.683012 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66d145-5c33-4126-bb83-747c6435c866 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.700610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba26c73-98f9-443e-b903-09d0e4ae1f85 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.733995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.734915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.735491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.736064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.736799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] RBAC: Adding query filter params (group.id=289105f07754455c9316a1ee7452c1e4, role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=de5700cfe30946bd8e0082d5b4964f98) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:01.737207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:01.740923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:01.742307 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836b5cb-edec-4b01-bbbc-150874d80bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.743171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836b5cb-edec-4b01-bbbc-150874d80bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.743636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836b5cb-edec-4b01-bbbc-150874d80bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.744043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836b5cb-edec-4b01-bbbc-150874d80bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.744922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836b5cb-edec-4b01-bbbc-150874d80bc2 None admin] RBAC: Adding query filter params (role.id=5681843b3fe64a0ebe56433cfca360e7, user.id=356ecba04f03492d97c5b5a093ff8781, scope.system=all) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:01.745375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836b5cb-edec-4b01-bbbc-150874d80bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:01.746989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836b5cb-edec-4b01-bbbc-150874d80bc2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:01.748906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:01.749190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 173/361] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/role_assignments?group.id=289105f07754455c9316a1ee7452c1e4&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:01.750596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.764034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 189/362] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/role_assignments?user.id=356ecba04f03492d97c5b5a093ff8781&scope.system=all&role.id=5681843b3fe64a0ebe56433cfca360e7 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:01.765249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657eba-91af-41a2-9a8b-5bce161eba9c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.765538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657eba-91af-41a2-9a8b-5bce161eba9c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.765809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657eba-91af-41a2-9a8b-5bce161eba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:01.785157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-31657eba-91af-41a2-9a8b-5bce161eba9c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:01.786380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-31657eba-91af-41a2-9a8b-5bce161eba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:01.861982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:01.899123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa9c-78ce-446c-8016-324dc3e711ee None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:01.914525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 190/363] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:01.915489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31657eba-91af-41a2-9a8b-5bce161eba9c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:01.974926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ea449-1231-49a3-bc14-83c6815f0330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:01.975612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ea449-1231-49a3-bc14-83c6815f0330 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:01.975890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ea449-1231-49a3-bc14-83c6815f0330 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:01.976160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ea449-1231-49a3-bc14-83c6815f0330 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:02.005879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ea449-1231-49a3-bc14-83c6815f0330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f00ea449-1231-49a3-bc14-83c6815f0330 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:02.013737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:02.014694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:02.015602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:02.017227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 174/364] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:02.022379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ea449-1231-49a3-bc14-83c6815f0330 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:02.123806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31657eba-91af-41a2-9a8b-5bce161eba9c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:02.174696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:02.196763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ea449-1231-49a3-bc14-83c6815f0330 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:02.198873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:02.199540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:02.199813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:02.200082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:02.200832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:02.201130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:02.202613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:02.220293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 191/365] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:05:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:02.232460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:02.275967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:02.276600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:02.276874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:02.277140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:02.277855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:02.278115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:02.279546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:02.297561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 175/366] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:02.309085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:02.336337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:02.366012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb4e20-32ae-4ac4-a660-0ba8171b8e66 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:02.404808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd1cb9-ac56-40a4-aaef-7e469dd7b6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:02.405356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd1cb9-ac56-40a4-aaef-7e469dd7b6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:02.405635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd1cb9-ac56-40a4-aaef-7e469dd7b6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:02.406219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd1cb9-ac56-40a4-aaef-7e469dd7b6a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:02.458890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93c9cb-c5b6-4037-bc64-186df2d06bb3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:02.462824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:02.495764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd1cb9-ac56-40a4-aaef-7e469dd7b6a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:02.497514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd1cb9-ac56-40a4-aaef-7e469dd7b6a5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:02.508417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115ce89-b9a2-4b53-aa82-a8b0777ceb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:02.509389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115ce89-b9a2-4b53-aa82-a8b0777ceb8f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:02.509854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115ce89-b9a2-4b53-aa82-a8b0777ceb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:02.510356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115ce89-b9a2-4b53-aa82-a8b0777ceb8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:02.601709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115ce89-b9a2-4b53-aa82-a8b0777ceb8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:02.603726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115ce89-b9a2-4b53-aa82-a8b0777ceb8f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:02.613372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 192/367] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:05:02.614505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd1cb9-ac56-40a4-aaef-7e469dd7b6a5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:02.619265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:02.712135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:02.718874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 176/368] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 12:05:02.719769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115ce89-b9a2-4b53-aa82-a8b0777ceb8f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:02.730317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd1cb9-ac56-40a4-aaef-7e469dd7b6a5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:02.772120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b72fb2-11d2-4fc5-bea4-ad0c4702cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:02.773104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b72fb2-11d2-4fc5-bea4-ad0c4702cf1b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:02.773610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b72fb2-11d2-4fc5-bea4-ad0c4702cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:02.774063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b72fb2-11d2-4fc5-bea4-ad0c4702cf1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:02.789308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b72fb2-11d2-4fc5-bea4-ad0c4702cf1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-05b72fb2-11d2-4fc5-bea4-ad0c4702cf1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:02.796592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:02.797533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:02.798305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:02.800842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 193/369] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:02.801480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b72fb2-11d2-4fc5-bea4-ad0c4702cf1b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:02.840445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115ce89-b9a2-4b53-aa82-a8b0777ceb8f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:02.856856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:02.899964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f0643-6662-4075-9c1c-9eda0f344bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:02.900261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f0643-6662-4075-9c1c-9eda0f344bfb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:02.900507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f0643-6662-4075-9c1c-9eda0f344bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:02.900780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f0643-6662-4075-9c1c-9eda0f344bfb None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:02.902116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f0643-6662-4075-9c1c-9eda0f344bfb None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:02.903448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f0643-6662-4075-9c1c-9eda0f344bfb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:02.905096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 177/370] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:02 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:02.905924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f0643-6662-4075-9c1c-9eda0f344bfb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:02.948010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b72fb2-11d2-4fc5-bea4-ad0c4702cf1b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:03.002669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedf8df-a67f-4ea3-b8e2-137ddaa9d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:03.002669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf8df-a67f-4ea3-b8e2-137ddaa9d630 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.002669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf8df-a67f-4ea3-b8e2-137ddaa9d630 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.003027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf8df-a67f-4ea3-b8e2-137ddaa9d630 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.005220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedf8df-a67f-4ea3-b8e2-137ddaa9d630 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:03.006702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedf8df-a67f-4ea3-b8e2-137ddaa9d630 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:03.008488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 194/371] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:02 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:03.009190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedf8df-a67f-4ea3-b8e2-137ddaa9d630 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:03.030797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:03.069203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f0643-6662-4075-9c1c-9eda0f344bfb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:03.110897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:03.111954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.112464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.112936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.113816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:03.114319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:03.115964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:03.126651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 178/372] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:03.127903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:03.157517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedf8df-a67f-4ea3-b8e2-137ddaa9d630 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:03.181637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:03.198730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004b300-c4dc-4732-a95b-c29fd9c2730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:03.199659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b300-c4dc-4732-a95b-c29fd9c2730f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.200103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b300-c4dc-4732-a95b-c29fd9c2730f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.200509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b300-c4dc-4732-a95b-c29fd9c2730f None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.201883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004b300-c4dc-4732-a95b-c29fd9c2730f None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:03.203561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004b300-c4dc-4732-a95b-c29fd9c2730f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:03.205317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 195/373] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:03 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:03.206374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004b300-c4dc-4732-a95b-c29fd9c2730f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:03.261462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa68d0-634f-4b92-928e-c413b9227ea0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:03.298987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:03.307177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe6b72-4f0f-4465-928f-620ae469d036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:03.307880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6b72-4f0f-4465-928f-620ae469d036 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.308160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6b72-4f0f-4465-928f-620ae469d036 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.308435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6b72-4f0f-4465-928f-620ae469d036 None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.310670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe6b72-4f0f-4465-928f-620ae469d036 None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:03.312552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe6b72-4f0f-4465-928f-620ae469d036 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:03.315371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 179/374] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:03 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:03.316448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe6b72-4f0f-4465-928f-620ae469d036 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:03.325170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004b300-c4dc-4732-a95b-c29fd9c2730f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:03.373114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61b97a-1a0d-4105-8052-96c357f1fce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:03.374088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61b97a-1a0d-4105-8052-96c357f1fce8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.374648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61b97a-1a0d-4105-8052-96c357f1fce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.375155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61b97a-1a0d-4105-8052-96c357f1fce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.462304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe6b72-4f0f-4465-928f-620ae469d036 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:03.470111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61b97a-1a0d-4105-8052-96c357f1fce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:03.471974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61b97a-1a0d-4105-8052-96c357f1fce8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:03.479688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:03.509686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:03.510660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.511125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.511705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.512623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] RBAC: Adding query filter params (group.id=289105f07754455c9316a1ee7452c1e4, role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:03.513149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:03.515882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:03.526014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 180/375] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:03 2022] GET /identity/v3/role_assignments?group.id=289105f07754455c9316a1ee7452c1e4&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:03.526914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:03.610692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:03.678337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 196/376] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:03.682694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e94c8-be79-46b7-891d-47fccd30e17d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:03.688400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61b97a-1a0d-4105-8052-96c357f1fce8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:03.732812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:03.733446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.733775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.734058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.734800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] RBAC: Adding query filter params (group.id=289105f07754455c9316a1ee7452c1e4, role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:03.735110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:03.740624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:03.750954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 181/377] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:03 2022] GET /identity/v3/role_assignments?group.id=289105f07754455c9316a1ee7452c1e4&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:03.855083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61b97a-1a0d-4105-8052-96c357f1fce8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:03.905378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5e5d1-af9e-4b8f-b9fd-0fa6ba7c36c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:03.906283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5e5d1-af9e-4b8f-b9fd-0fa6ba7c36c7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.906571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5e5d1-af9e-4b8f-b9fd-0fa6ba7c36c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.907047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5e5d1-af9e-4b8f-b9fd-0fa6ba7c36c7 None admin] PATH_INFO: `/v3/projects/b6117aaa4eb34295b2216e51a40eacde` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.908961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5e5d1-af9e-4b8f-b9fd-0fa6ba7c36c7 None admin] RBAC: Authorizing `identity:get_project(project_id=b6117aaa4eb34295b2216e51a40eacde)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:03.910460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5e5d1-af9e-4b8f-b9fd-0fa6ba7c36c7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:03.912233 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 197/378] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:03 2022] GET /identity/v3/projects/b6117aaa4eb34295b2216e51a40eacde => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:03.919019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:03.940980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e007189-9c57-41aa-aed0-813cf26aafe2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.941293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e007189-9c57-41aa-aed0-813cf26aafe2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.941564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e007189-9c57-41aa-aed0-813cf26aafe2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.943254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 198/379] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:03.953231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3bbb1-3d38-425a-acab-4b6f90e5fb2b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:03.953535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3bbb1-3d38-425a-acab-4b6f90e5fb2b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:03.953790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3bbb1-3d38-425a-acab-4b6f90e5fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:03.972125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:03.980922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c3a3bbb1-3d38-425a-acab-4b6f90e5fb2b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:03.982141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c3a3bbb1-3d38-425a-acab-4b6f90e5fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:04.047408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b74926-5492-40a8-8cf4-783410ce4797 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:04.061047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:04.111459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 199/380] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:05:04.115955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ef6e-4031-497b-a444-418e8e0375a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:04.117318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ef6e-4031-497b-a444-418e8e0375a0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:04.117824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ef6e-4031-497b-a444-418e8e0375a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:04.118157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ef6e-4031-497b-a444-418e8e0375a0 None admin] PATH_INFO: `/v3/groups/6ab967463d8b458a89c6215291537c9f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:04.124460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ef6e-4031-497b-a444-418e8e0375a0 None admin] RBAC: Authorizing `identity:get_group(group_id=6ab967463d8b458a89c6215291537c9f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:04.126399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ef6e-4031-497b-a444-418e8e0375a0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:04.129885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 182/381] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:03 2022] GET /identity/v3/groups/6ab967463d8b458a89c6215291537c9f => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:04.137798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3bbb1-3d38-425a-acab-4b6f90e5fb2b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:04.272704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb4a47-605f-4075-84e4-87b26feba675 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:04.273367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb4a47-605f-4075-84e4-87b26feba675 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:04.273833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb4a47-605f-4075-84e4-87b26feba675 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:04.275763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 183/382] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:05:04.281718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b633a-68f6-43e0-ae87-276aac93eeab None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:04.282352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b633a-68f6-43e0-ae87-276aac93eeab None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:04.282813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b633a-68f6-43e0-ae87-276aac93eeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:04.306335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-393b633a-68f6-43e0-ae87-276aac93eeab None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:04.308059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-393b633a-68f6-43e0-ae87-276aac93eeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:04.314685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3bbb1-3d38-425a-acab-4b6f90e5fb2b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:04.389077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:04.390238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:04.390804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:04.391317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:04.392279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] RBAC: Adding query filter params (group.id=6ab967463d8b458a89c6215291537c9f, role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=b6117aaa4eb34295b2216e51a40eacde) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:04.392783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:04.394432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:04.405720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 200/383] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:04 2022] GET /identity/v3/role_assignments?group.id=6ab967463d8b458a89c6215291537c9f&scope.project.id=b6117aaa4eb34295b2216e51a40eacde&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:04.429101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 184/384] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:04.442276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:04.496421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:04.561760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef062fa-2e76-4ea5-bccb-f6b010ad04df None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:04.605461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89077a-6aa1-46f3-9ca4-672e92a2fc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:04.606145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89077a-6aa1-46f3-9ca4-672e92a2fc67 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:04.606414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89077a-6aa1-46f3-9ca4-672e92a2fc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:04.606680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89077a-6aa1-46f3-9ca4-672e92a2fc67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:04.624342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89077a-6aa1-46f3-9ca4-672e92a2fc67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-4d89077a-6aa1-46f3-9ca4-672e92a2fc67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:04.629831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:04.630753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:04.631595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:04.632679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 201/385] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:04.637388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b633a-68f6-43e0-ae87-276aac93eeab None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:04.668136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:04.762638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b633a-68f6-43e0-ae87-276aac93eeab None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:04.785876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89077a-6aa1-46f3-9ca4-672e92a2fc67 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:04.812087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:04.813165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:04.813949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:04.815500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:04.816415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:04.816920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:04.818607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:04.830870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 185/386] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:04.841584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:04.950135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89077a-6aa1-46f3-9ca4-672e92a2fc67 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:04.962214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:04.995478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1027c-2178-4f01-a852-730c5a8419bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:04.996225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1027c-2178-4f01-a852-730c5a8419bc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:04.996565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1027c-2178-4f01-a852-730c5a8419bc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:04.996898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1027c-2178-4f01-a852-730c5a8419bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.015869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1027c-2178-4f01-a852-730c5a8419bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-34b1027c-2178-4f01-a852-730c5a8419bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:05.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:05.025252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:05.026041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:05.026651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:05.029485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 202/387] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:05.035501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1027c-2178-4f01-a852-730c5a8419bc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:05.046222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41f117-8633-447a-8b46-e92bc2a5dd69 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:05.144449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1027c-2178-4f01-a852-730c5a8419bc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:05.148593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69ed4e-07f9-43bb-b13a-204442b6f04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:05.149257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69ed4e-07f9-43bb-b13a-204442b6f04d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.149537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69ed4e-07f9-43bb-b13a-204442b6f04d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.149811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69ed4e-07f9-43bb-b13a-204442b6f04d None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.150946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69ed4e-07f9-43bb-b13a-204442b6f04d None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:05.152451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69ed4e-07f9-43bb-b13a-204442b6f04d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:05.154506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 186/388] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:04 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:05.172400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69ed4e-07f9-43bb-b13a-204442b6f04d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:05.186706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:05.187648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.187951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.188347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.189111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:05.189418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:05.190978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:05.201129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 203/389] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:05:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:05.207284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:05.263761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:05.395882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69ed4e-07f9-43bb-b13a-204442b6f04d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:05.419796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b38ee-6c11-4fd8-a446-e8b1cb52f999 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:05.466920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ad061-b272-4d06-ba53-582790bc3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:05.467611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad061-b272-4d06-ba53-582790bc3b08 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.468474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad061-b272-4d06-ba53-582790bc3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.468749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad061-b272-4d06-ba53-582790bc3b08 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.469927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0da7d-f342-4787-9380-b81e303576f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:05.470566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0da7d-f342-4787-9380-b81e303576f6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.470831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0da7d-f342-4787-9380-b81e303576f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.471094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0da7d-f342-4787-9380-b81e303576f6 None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.472241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0da7d-f342-4787-9380-b81e303576f6 None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:05.473695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0da7d-f342-4787-9380-b81e303576f6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:05.475658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 204/390] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:05 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:05.476030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ad061-b272-4d06-ba53-582790bc3b08 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:05.477530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ad061-b272-4d06-ba53-582790bc3b08 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:05.479436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 187/391] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:05 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:05.485198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0da7d-f342-4787-9380-b81e303576f6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:05.486026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ad061-b272-4d06-ba53-582790bc3b08 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:05.557219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:05.623798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:05.649252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0da7d-f342-4787-9380-b81e303576f6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:05.667104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ad061-b272-4d06-ba53-582790bc3b08 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:05.705785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df027-a4be-4eaa-a349-4a3b6ceb2e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:05.707055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df027-a4be-4eaa-a349-4a3b6ceb2e12 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.707528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df027-a4be-4eaa-a349-4a3b6ceb2e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.707967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df027-a4be-4eaa-a349-4a3b6ceb2e12 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.709317 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4df027-a4be-4eaa-a349-4a3b6ceb2e12 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:05.711559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4df027-a4be-4eaa-a349-4a3b6ceb2e12 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:05.713815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 188/392] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:05 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:05.719079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df027-a4be-4eaa-a349-4a3b6ceb2e12 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:05.733368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b4643-3414-4f3f-8763-67d237c0dc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:05.734114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b4643-3414-4f3f-8763-67d237c0dc19 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.734513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b4643-3414-4f3f-8763-67d237c0dc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.734855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b4643-3414-4f3f-8763-67d237c0dc19 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98/groups/289105f07754455c9316a1ee7452c1e4/roles/0f4eb127111b4fdabec35268db5bde9e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.741316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b4643-3414-4f3f-8763-67d237c0dc19 None admin] RBAC: Authorizing `identity:create_grant(project_id=de5700cfe30946bd8e0082d5b4964f98, group_id=289105f07754455c9316a1ee7452c1e4, role_id=0f4eb127111b4fdabec35268db5bde9e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:05.742763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b4643-3414-4f3f-8763-67d237c0dc19 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:05.782183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 205/393] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 31 12:05:05 2022] PUT /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98/groups/289105f07754455c9316a1ee7452c1e4/roles/0f4eb127111b4fdabec35268db5bde9e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:05.885298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:05.899268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df027-a4be-4eaa-a349-4a3b6ceb2e12 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:05.918542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d47e2-35d8-44fe-82a1-46cf6a79b2aa None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.918883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d47e2-35d8-44fe-82a1-46cf6a79b2aa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.919096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d47e2-35d8-44fe-82a1-46cf6a79b2aa None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.920839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 206/394] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:05.926450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada8a69-b226-47fa-8343-2c1b3829e8c2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.926756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada8a69-b226-47fa-8343-2c1b3829e8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.926996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada8a69-b226-47fa-8343-2c1b3829e8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.941625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db93b66-87e8-4b04-9598-fffde5f9668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:05.942688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db93b66-87e8-4b04-9598-fffde5f9668e None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:05.943165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db93b66-87e8-4b04-9598-fffde5f9668e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:05.943661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db93b66-87e8-4b04-9598-fffde5f9668e None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98/groups/289105f07754455c9316a1ee7452c1e4/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:05.947827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db93b66-87e8-4b04-9598-fffde5f9668e None admin] RBAC: Authorizing `identity:create_grant(project_id=de5700cfe30946bd8e0082d5b4964f98, group_id=289105f07754455c9316a1ee7452c1e4, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:05.949580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db93b66-87e8-4b04-9598-fffde5f9668e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:05.956441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6ada8a69-b226-47fa-8343-2c1b3829e8c2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:05.960641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6ada8a69-b226-47fa-8343-2c1b3829e8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:05.978093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 189/395] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 31 12:05:05 2022] PUT /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98/groups/289105f07754455c9316a1ee7452c1e4/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:06.061255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 207/396] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:06.355468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db93b66-87e8-4b04-9598-fffde5f9668e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:06.462226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe93d75-4a16-4693-8c76-33db467f3c1e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:06.463253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe93d75-4a16-4693-8c76-33db467f3c1e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:06.463736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe93d75-4a16-4693-8c76-33db467f3c1e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:06.465584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 208/397] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:06.471663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dc201-7f41-45ea-b495-740a67942636 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:06.472257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dc201-7f41-45ea-b495-740a67942636 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:06.472748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dc201-7f41-45ea-b495-740a67942636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:06.483069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db93b66-87e8-4b04-9598-fffde5f9668e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:06.494315 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e78dc201-7f41-45ea-b495-740a67942636 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:06.495876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e78dc201-7f41-45ea-b495-740a67942636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:06.550150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb1762-b9fd-49f2-bbcd-2b0fd4fce2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:06.551187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb1762-b9fd-49f2-bbcd-2b0fd4fce2eb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:06.551707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb1762-b9fd-49f2-bbcd-2b0fd4fce2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:06.555716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb1762-b9fd-49f2-bbcd-2b0fd4fce2eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:06.582342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb1762-b9fd-49f2-bbcd-2b0fd4fce2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:06.588619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 209/398] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ddfb1762-b9fd-49f2-bbcd-2b0fd4fce2eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:06.591854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:06.592773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:06.594261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 190/399] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:06.601655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dc201-7f41-45ea-b495-740a67942636 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:06.629797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:06.803669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dc201-7f41-45ea-b495-740a67942636 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:06.850673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:06.851321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:06.851602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:06.851874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:06.852606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:06.852908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:06.854349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:06.867321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 210/400] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:05:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:06.873034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb1762-b9fd-49f2-bbcd-2b0fd4fce2eb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:06.904399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:07.008463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb1762-b9fd-49f2-bbcd-2b0fd4fce2eb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:07.049769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:07.050811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3588e-cee2-4d70-840d-c470101c6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:07.051485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3588e-cee2-4d70-840d-c470101c6766 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.051763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3588e-cee2-4d70-840d-c470101c6766 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.052033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3588e-cee2-4d70-840d-c470101c6766 None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.053162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3588e-cee2-4d70-840d-c470101c6766 None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:07.054634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3588e-cee2-4d70-840d-c470101c6766 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:07.056271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 191/401] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:06 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:07.063658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3588e-cee2-4d70-840d-c470101c6766 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:07.161339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:07.166054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:07.181784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3588e-cee2-4d70-840d-c470101c6766 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:07.209220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8146797-7fc3-46e9-9ca4-19aec857d786 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:07.223255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eaa1fd-c93b-4ba5-a91b-029c1a834803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:07.224551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa1fd-c93b-4ba5-a91b-029c1a834803 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.225017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa1fd-c93b-4ba5-a91b-029c1a834803 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.225469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa1fd-c93b-4ba5-a91b-029c1a834803 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.226778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eaa1fd-c93b-4ba5-a91b-029c1a834803 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:07.228472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eaa1fd-c93b-4ba5-a91b-029c1a834803 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:07.235354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 192/402] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:07 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:07.237016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17814748-9a79-4f5a-8ff5-5b1b36e3e675 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.237490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17814748-9a79-4f5a-8ff5-5b1b36e3e675 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.237949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17814748-9a79-4f5a-8ff5-5b1b36e3e675 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.240503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 193/403] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:07.241519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17814748-9a79-4f5a-8ff5-5b1b36e3e675 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:07.255409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa215a5b-63a4-44c4-aca7-5174074fafa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:07.256069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa215a5b-63a4-44c4-aca7-5174074fafa9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.256406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa215a5b-63a4-44c4-aca7-5174074fafa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.256680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa215a5b-63a4-44c4-aca7-5174074fafa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.271962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa215a5b-63a4-44c4-aca7-5174074fafa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fa215a5b-63a4-44c4-aca7-5174074fafa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:07.279797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:07.280883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:07.281683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:07.284456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 211/404] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:07.286216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda5ac6-ea91-4ffc-b148-24adef19cbf8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.286560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda5ac6-ea91-4ffc-b148-24adef19cbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.288917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda5ac6-ea91-4ffc-b148-24adef19cbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.312952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3cda5ac6-ea91-4ffc-b148-24adef19cbf8 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:07.314596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3cda5ac6-ea91-4ffc-b148-24adef19cbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:07.362412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17814748-9a79-4f5a-8ff5-5b1b36e3e675 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:07.391459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 212/405] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:07.392625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda5ac6-ea91-4ffc-b148-24adef19cbf8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:07.440675 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a4f81-7bcd-4ba0-94fd-4f2d5219db45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:07.441302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a4f81-7bcd-4ba0-94fd-4f2d5219db45 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.441572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a4f81-7bcd-4ba0-94fd-4f2d5219db45 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.441844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a4f81-7bcd-4ba0-94fd-4f2d5219db45 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/groups/289105f07754455c9316a1ee7452c1e4/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.443763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a4f81-7bcd-4ba0-94fd-4f2d5219db45 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb8bbbfbab8e46d49858d9ef99a96374, group_id=289105f07754455c9316a1ee7452c1e4, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:07.445224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a4f81-7bcd-4ba0-94fd-4f2d5219db45 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:07.469301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:07.486394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 194/406] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 31 12:05:07 2022] PUT /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/groups/289105f07754455c9316a1ee7452c1e4/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:07.508236 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda5ac6-ea91-4ffc-b148-24adef19cbf8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:07.545652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:07.546805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.547311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.547771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.548639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:07.549126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:07.550723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:07.560327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 213/407] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:07.566378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a4f81-7bcd-4ba0-94fd-4f2d5219db45 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:07.583804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:07.697616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a4f81-7bcd-4ba0-94fd-4f2d5219db45 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:07.733151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f45040-a909-4bad-9d42-34b5c79afc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:07.733951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f45040-a909-4bad-9d42-34b5c79afc4e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.734297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f45040-a909-4bad-9d42-34b5c79afc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.734647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f45040-a909-4bad-9d42-34b5c79afc4e None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.735944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f45040-a909-4bad-9d42-34b5c79afc4e None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:07.737470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f45040-a909-4bad-9d42-34b5c79afc4e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:07.739546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 195/408] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:07 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:07.745160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:07.765603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f45040-a909-4bad-9d42-34b5c79afc4e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:07.851408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:07.856677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a357c21-e0df-4405-9df3-11a068a6d9f7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:07.876659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f45040-a909-4bad-9d42-34b5c79afc4e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:07.903615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:07.906653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443b25-99cb-4d9a-9c60-b8d8bd923d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:07.907477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443b25-99cb-4d9a-9c60-b8d8bd923d00 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.907833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443b25-99cb-4d9a-9c60-b8d8bd923d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.908177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443b25-99cb-4d9a-9c60-b8d8bd923d00 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.909610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49443b25-99cb-4d9a-9c60-b8d8bd923d00 None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:07.911310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49443b25-99cb-4d9a-9c60-b8d8bd923d00 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:07.913692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 214/409] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:07 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:07.920804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443b25-99cb-4d9a-9c60-b8d8bd923d00 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:07.928740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15aced-5a20-4041-815f-d7677581a571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:07.930000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15aced-5a20-4041-815f-d7677581a571 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:07.930687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15aced-5a20-4041-815f-d7677581a571 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:07.931289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15aced-5a20-4041-815f-d7677581a571 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:07.950376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15aced-5a20-4041-815f-d7677581a571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ff15aced-5a20-4041-815f-d7677581a571 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:07.960998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:07.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:07.964563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:07.974339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 196/410] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:07.978701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15aced-5a20-4041-815f-d7677581a571 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:08.041625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443b25-99cb-4d9a-9c60-b8d8bd923d00 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:08.081402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d2e08-0c22-4545-80e4-9b27668ec915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:08.082247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d2e08-0c22-4545-80e4-9b27668ec915 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.083123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d2e08-0c22-4545-80e4-9b27668ec915 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.083523 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d2e08-0c22-4545-80e4-9b27668ec915 None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/groups/289105f07754455c9316a1ee7452c1e4/roles/0f4eb127111b4fdabec35268db5bde9e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.091417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d2e08-0c22-4545-80e4-9b27668ec915 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb8bbbfbab8e46d49858d9ef99a96374, group_id=289105f07754455c9316a1ee7452c1e4, role_id=0f4eb127111b4fdabec35268db5bde9e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:08.094072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d2e08-0c22-4545-80e4-9b27668ec915 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:08.124208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 215/411] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 31 12:05:07 2022] PUT /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374/groups/289105f07754455c9316a1ee7452c1e4/roles/0f4eb127111b4fdabec35268db5bde9e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:05:08.193795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15aced-5a20-4041-815f-d7677581a571 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:08.232557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:08.233179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.233482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.233914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.234752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:08.235134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:08.236616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:08.243125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:08.248532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 197/412] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:08.255900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d2e08-0c22-4545-80e4-9b27668ec915 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:08.361873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:08.409321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d2e08-0c22-4545-80e4-9b27668ec915 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:08.451167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8914983-72eb-4ace-9eee-b00e31b55bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:08.451858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8914983-72eb-4ace-9eee-b00e31b55bef None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.452134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8914983-72eb-4ace-9eee-b00e31b55bef None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.452556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8914983-72eb-4ace-9eee-b00e31b55bef None admin] PATH_INFO: `/v3/groups/6ab967463d8b458a89c6215291537c9f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.453707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8914983-72eb-4ace-9eee-b00e31b55bef None admin] RBAC: Authorizing `identity:get_group(group_id=6ab967463d8b458a89c6215291537c9f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:08.455200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8914983-72eb-4ace-9eee-b00e31b55bef None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:08.457139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 216/413] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:08 2022] GET /identity/v3/groups/6ab967463d8b458a89c6215291537c9f => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:08.462799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:08.472946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80203b06-b48d-4088-bd08-186b4ba47cd5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.474182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80203b06-b48d-4088-bd08-186b4ba47cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.474661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80203b06-b48d-4088-bd08-186b4ba47cd5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.477674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 217/414] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:05:08.482514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b838-ed8f-4035-85be-f03e8c182b1a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.482855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b838-ed8f-4035-85be-f03e8c182b1a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.483075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b838-ed8f-4035-85be-f03e8c182b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.503848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5511b838-ed8f-4035-85be-f03e8c182b1a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:08.505390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5511b838-ed8f-4035-85be-f03e8c182b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:08.592122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d2b54-30f2-4de8-8df7-eb124ebb3228 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:08.593211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 218/415] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:05:08.594543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b5e21-bafc-4858-9f3e-f63ed7fec067 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.594827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b5e21-bafc-4858-9f3e-f63ed7fec067 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.595418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b5e21-bafc-4858-9f3e-f63ed7fec067 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.597726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 219/416] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:08.607952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964b425-c357-43db-8941-1c80e16df023 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.611218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964b425-c357-43db-8941-1c80e16df023 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.611821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964b425-c357-43db-8941-1c80e16df023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.646525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7964b425-c357-43db-8941-1c80e16df023 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:08.648852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7964b425-c357-43db-8941-1c80e16df023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:08.662994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ebf8a-bc37-4945-b8bc-1aeff13b7af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:08.664046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ebf8a-bc37-4945-b8bc-1aeff13b7af6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.664516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ebf8a-bc37-4945-b8bc-1aeff13b7af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.665286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ebf8a-bc37-4945-b8bc-1aeff13b7af6 None admin] PATH_INFO: `/v3/projects/b6117aaa4eb34295b2216e51a40eacde` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.667498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ebf8a-bc37-4945-b8bc-1aeff13b7af6 None admin] RBAC: Authorizing `identity:get_project(project_id=b6117aaa4eb34295b2216e51a40eacde)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:08.670688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ebf8a-bc37-4945-b8bc-1aeff13b7af6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:08.676157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 198/417] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:08 2022] GET /identity/v3/projects/b6117aaa4eb34295b2216e51a40eacde => generated 344 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:08.681075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ebf8a-bc37-4945-b8bc-1aeff13b7af6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:08.748468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:08.785525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 220/418] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:05:08.807820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:08.870018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ebf8a-bc37-4945-b8bc-1aeff13b7af6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:08.918799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964b425-c357-43db-8941-1c80e16df023 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:08.921810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4a885-35c9-4213-bc87-dd21c5e612ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:08.922598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4a885-35c9-4213-bc87-dd21c5e612ac None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:08.922866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4a885-35c9-4213-bc87-dd21c5e612ac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:08.923136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4a885-35c9-4213-bc87-dd21c5e612ac None admin] PATH_INFO: `/v3/projects/b6117aaa4eb34295b2216e51a40eacde/groups/6ab967463d8b458a89c6215291537c9f/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:08.927875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4a885-35c9-4213-bc87-dd21c5e612ac None admin] RBAC: Authorizing `identity:create_grant(project_id=b6117aaa4eb34295b2216e51a40eacde, group_id=6ab967463d8b458a89c6215291537c9f, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:08.929349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4a885-35c9-4213-bc87-dd21c5e612ac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:08.953790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 199/419] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 31 12:05:08 2022] PUT /identity/v3/projects/b6117aaa4eb34295b2216e51a40eacde/groups/6ab967463d8b458a89c6215291537c9f/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:09.052928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964b425-c357-43db-8941-1c80e16df023 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:09.090635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dce3e-5b40-4bd0-b37b-a11965f002cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:09.091291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dce3e-5b40-4bd0-b37b-a11965f002cf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:09.091571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dce3e-5b40-4bd0-b37b-a11965f002cf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:09.091843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dce3e-5b40-4bd0-b37b-a11965f002cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:09.104081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808dce3e-5b40-4bd0-b37b-a11965f002cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-808dce3e-5b40-4bd0-b37b-a11965f002cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:09.109383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:09.110256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:09.111049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 221/420] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:09.118213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4a885-35c9-4213-bc87-dd21c5e612ac None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:09.150308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dce3e-5b40-4bd0-b37b-a11965f002cf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:09.209963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:09.272889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4a885-35c9-4213-bc87-dd21c5e612ac None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:09.287651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dce3e-5b40-4bd0-b37b-a11965f002cf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:09.312022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:09.331188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:09.332240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:09.332712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:09.333183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:09.334049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:09.335042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:09.337180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:09.342123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11febe4-fa75-4905-b5fe-a76a8f50f6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:09.343519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11febe4-fa75-4905-b5fe-a76a8f50f6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:09.343995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11febe4-fa75-4905-b5fe-a76a8f50f6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:09.344712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11febe4-fa75-4905-b5fe-a76a8f50f6b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:09.358323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 200/421] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:09.365694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:09.380525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11febe4-fa75-4905-b5fe-a76a8f50f6b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b11febe4-fa75-4905-b5fe-a76a8f50f6b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:09.385755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:09.386723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:09.387529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:09.390011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 222/422] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:09.396276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11febe4-fa75-4905-b5fe-a76a8f50f6b7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:09.580532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11febe4-fa75-4905-b5fe-a76a8f50f6b7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:09.583032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:09.611485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e48c2-e03c-4d44-b461-f95828070515 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:09.653723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8d967-d23a-4fbd-b5fe-7fe20381bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:09.654715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:09.655994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:09.656840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8d967-d23a-4fbd-b5fe-7fe20381bf92 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:09.657313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8d967-d23a-4fbd-b5fe-7fe20381bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:09.657792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8d967-d23a-4fbd-b5fe-7fe20381bf92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:09.658265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:09.659337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:09.660216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:09.661047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:09.664521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:09.682717 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 223/423] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:05:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:09.690111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:09.777778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:09.795225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8d967-d23a-4fbd-b5fe-7fe20381bf92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:09.797126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8d967-d23a-4fbd-b5fe-7fe20381bf92 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:09.886268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17c509-5a77-48a7-81ae-5ffd9b26f1db None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:09.935998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785dd90-628b-4a80-a83c-7caf86ad7bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:09.936809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785dd90-628b-4a80-a83c-7caf86ad7bae None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:09.937223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785dd90-628b-4a80-a83c-7caf86ad7bae None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:09.937748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785dd90-628b-4a80-a83c-7caf86ad7bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:09.953139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 201/424] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:09.954609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32bc3a-d408-4545-9749-a12045232ab9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:09.954892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32bc3a-d408-4545-9749-a12045232ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:09.955163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32bc3a-d408-4545-9749-a12045232ab9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:09.956718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 202/425] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:09.959617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32bc3a-d408-4545-9749-a12045232ab9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:09.986011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:10.039940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785dd90-628b-4a80-a83c-7caf86ad7bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:10.041595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785dd90-628b-4a80-a83c-7caf86ad7bae None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:10.060684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:10.072304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32bc3a-d408-4545-9749-a12045232ab9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:10.129826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69927c4-70c8-44e8-ae52-da5531c88120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:10.131052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69927c4-70c8-44e8-ae52-da5531c88120 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.131698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69927c4-70c8-44e8-ae52-da5531c88120 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.132338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69927c4-70c8-44e8-ae52-da5531c88120 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.134394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69927c4-70c8-44e8-ae52-da5531c88120 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:10.137104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69927c4-70c8-44e8-ae52-da5531c88120 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:10.141204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 203/426] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:09 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:10.143489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9267-7fd7-4919-ad48-ad639814ae52 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.144114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9267-7fd7-4919-ad48-ad639814ae52 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.144710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9267-7fd7-4919-ad48-ad639814ae52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.171349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 224/427] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:10.172339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785dd90-628b-4a80-a83c-7caf86ad7bae None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:10.178682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-db9e9267-7fd7-4919-ad48-ad639814ae52 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:10.179913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db9e9267-7fd7-4919-ad48-ad639814ae52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:10.263260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 204/428] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:10.263260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9267-7fd7-4919-ad48-ad639814ae52 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:10.263815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:10.333172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785dd90-628b-4a80-a83c-7caf86ad7bae None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:10.375686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabe4e4-c6d0-42cc-9459-60340db44f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:10.376726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabe4e4-c6d0-42cc-9459-60340db44f5e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.377345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabe4e4-c6d0-42cc-9459-60340db44f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.377831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabe4e4-c6d0-42cc-9459-60340db44f5e None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.379176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabe4e4-c6d0-42cc-9459-60340db44f5e None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:10.381254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabe4e4-c6d0-42cc-9459-60340db44f5e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:10.383164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 225/429] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:10 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:10.389155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabe4e4-c6d0-42cc-9459-60340db44f5e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:10.431958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:10.444931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9267-7fd7-4919-ad48-ad639814ae52 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:10.480388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf33d4-3cbf-495d-97bf-d7632de44ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:10.481032 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf33d4-3cbf-495d-97bf-d7632de44ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.481308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf33d4-3cbf-495d-97bf-d7632de44ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.481579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf33d4-3cbf-495d-97bf-d7632de44ab1 None admin] PATH_INFO: `/v3/projects/de5700cfe30946bd8e0082d5b4964f98` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.482712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf33d4-3cbf-495d-97bf-d7632de44ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=de5700cfe30946bd8e0082d5b4964f98)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:10.484254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf33d4-3cbf-495d-97bf-d7632de44ab1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:10.486340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 205/430] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:10 2022] GET /identity/v3/projects/de5700cfe30946bd8e0082d5b4964f98 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:10.491482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518826d4-efce-401b-8d6d-a9d8ffc40869 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.491482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518826d4-efce-401b-8d6d-a9d8ffc40869 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.491482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518826d4-efce-401b-8d6d-a9d8ffc40869 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.493386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 206/431] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:10.494191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518826d4-efce-401b-8d6d-a9d8ffc40869 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:10.520078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabe4e4-c6d0-42cc-9459-60340db44f5e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:10.554287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:10.561187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10c288-e62d-4ada-8736-bb5ec0cfee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:10.562472 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10c288-e62d-4ada-8736-bb5ec0cfee46 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.563056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10c288-e62d-4ada-8736-bb5ec0cfee46 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.563629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10c288-e62d-4ada-8736-bb5ec0cfee46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.564728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10c288-e62d-4ada-8736-bb5ec0cfee46 None admin] RBAC: Adding query filter params (group.id=289105f07754455c9316a1ee7452c1e4, role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=de5700cfe30946bd8e0082d5b4964f98) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:10.565234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10c288-e62d-4ada-8736-bb5ec0cfee46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:10.566929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10c288-e62d-4ada-8736-bb5ec0cfee46 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:10.574202 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 226/432] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:10 2022] GET /identity/v3/role_assignments?group.id=289105f07754455c9316a1ee7452c1e4&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:10.575971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8858-37b4-4ee9-a871-558651324ca4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.576454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8858-37b4-4ee9-a871-558651324ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.576918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8858-37b4-4ee9-a871-558651324ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.598954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-372b8858-37b4-4ee9-a871-558651324ca4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:10.601119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-372b8858-37b4-4ee9-a871-558651324ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:10.613782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518826d4-efce-401b-8d6d-a9d8ffc40869 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:10.653791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc60ce-0080-42e0-84fe-4b9389e8ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:10.654354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc60ce-0080-42e0-84fe-4b9389e8ef9e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.654625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc60ce-0080-42e0-84fe-4b9389e8ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.654890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc60ce-0080-42e0-84fe-4b9389e8ef9e None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.656321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc60ce-0080-42e0-84fe-4b9389e8ef9e None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:10.658158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc60ce-0080-42e0-84fe-4b9389e8ef9e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:10.663273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 207/433] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:10 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:10.663273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc60ce-0080-42e0-84fe-4b9389e8ef9e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:10.698744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 227/434] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:10.700297 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8858-37b4-4ee9-a871-558651324ca4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:10.705601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:10.811836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc60ce-0080-42e0-84fe-4b9389e8ef9e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:10.837846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8858-37b4-4ee9-a871-558651324ca4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:10.855918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2072b-3cec-4515-8e11-811304ed68f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:10.856705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2072b-3cec-4515-8e11-811304ed68f7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.857127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2072b-3cec-4515-8e11-811304ed68f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.857463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2072b-3cec-4515-8e11-811304ed68f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.874893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2072b-3cec-4515-8e11-811304ed68f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:10.877218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-06d2072b-3cec-4515-8e11-811304ed68f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:10.882095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:10.883223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 12:05:10.884172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:10.886314 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 208/435] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:10.895455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2072b-3cec-4515-8e11-811304ed68f7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:10.897115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:10.897985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:10.898451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:10.899293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:10.901259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] RBAC: Adding query filter params (group.id=289105f07754455c9316a1ee7452c1e4, role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=de5700cfe30946bd8e0082d5b4964f98) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:10.901767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:10.903548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:10.912879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 228/436] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:10 2022] GET /identity/v3/role_assignments?group.id=289105f07754455c9316a1ee7452c1e4&scope.project.id=de5700cfe30946bd8e0082d5b4964f98&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:11.001219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:11.021007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2072b-3cec-4515-8e11-811304ed68f7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:11.058648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:11.059364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.059640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.059907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.060647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:11.060946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:11.062511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:11.072354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 209/437] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:05:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:11.074555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.083516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.197517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864ea6-3d2d-436a-a83c-86e722b6161c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:11.213345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad6ce0-5796-41ae-86bc-b0e755bbacb2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:11.245745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8012d-c206-410b-a9fc-c2392c9a5c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:11.246386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8012d-c206-410b-a9fc-c2392c9a5c14 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.247644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8012d-c206-410b-a9fc-c2392c9a5c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.247934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8012d-c206-410b-a9fc-c2392c9a5c14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.256354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c55a9-e6a7-46f4-8d6a-45f930935f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:11.257465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c55a9-e6a7-46f4-8d6a-45f930935f06 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.257465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c55a9-e6a7-46f4-8d6a-45f930935f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.257657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c55a9-e6a7-46f4-8d6a-45f930935f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.259322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8012d-c206-410b-a9fc-c2392c9a5c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8da8012d-c206-410b-a9fc-c2392c9a5c14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:11.264665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:11.266013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:11.266811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 229/438] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:11.267687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae10738-9887-47d1-ae1c-de1e195173be None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.267852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae10738-9887-47d1-ae1c-de1e195173be None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.268121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae10738-9887-47d1-ae1c-de1e195173be None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.269375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 230/439] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:11.274185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae10738-9887-47d1-ae1c-de1e195173be None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.278740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:11.332751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:11.341218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c55a9-e6a7-46f4-8d6a-45f930935f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:11.342719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c55a9-e6a7-46f4-8d6a-45f930935f06 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:11.404916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae10738-9887-47d1-ae1c-de1e195173be None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:11.447726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b317ca-ccb4-49df-992f-0021f3b9d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:11.448270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b317ca-ccb4-49df-992f-0021f3b9d91f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.448542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b317ca-ccb4-49df-992f-0021f3b9d91f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.449537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b317ca-ccb4-49df-992f-0021f3b9d91f None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.449537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b317ca-ccb4-49df-992f-0021f3b9d91f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:11.449778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b317ca-ccb4-49df-992f-0021f3b9d91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:11.451575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b317ca-ccb4-49df-992f-0021f3b9d91f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:11.460844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 231/440] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:11.462109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff6c28-518f-477f-a1d2-f7495fedb528 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.462394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff6c28-518f-477f-a1d2-f7495fedb528 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.462666 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff6c28-518f-477f-a1d2-f7495fedb528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.467426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 210/441] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:11.469618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c55a9-e6a7-46f4-8d6a-45f930935f06 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.484522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1dff6c28-518f-477f-a1d2-f7495fedb528 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:11.485748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1dff6c28-518f-477f-a1d2-f7495fedb528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:11.525233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:11.560288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 232/442] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:11.561135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff6c28-518f-477f-a1d2-f7495fedb528 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.580209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c55a9-e6a7-46f4-8d6a-45f930935f06 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:11.614334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:11.617341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c60563-35ad-4080-98f8-bd22557c2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:11.617952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60563-35ad-4080-98f8-bd22557c2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.618224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60563-35ad-4080-98f8-bd22557c2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.618488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60563-35ad-4080-98f8-bd22557c2cb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.681343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff6c28-518f-477f-a1d2-f7495fedb528 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:11.687858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c60563-35ad-4080-98f8-bd22557c2cb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:11.689588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c60563-35ad-4080-98f8-bd22557c2cb5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:11.721698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619f09b-54e7-4715-8d01-dc9f63dce24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:11.722371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619f09b-54e7-4715-8d01-dc9f63dce24c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.722647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619f09b-54e7-4715-8d01-dc9f63dce24c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.722911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619f09b-54e7-4715-8d01-dc9f63dce24c None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.724412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619f09b-54e7-4715-8d01-dc9f63dce24c None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:11.725860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619f09b-54e7-4715-8d01-dc9f63dce24c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:11.727537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 233/443] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:11.733028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619f09b-54e7-4715-8d01-dc9f63dce24c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.740429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:11.802064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 211/444] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:11.807414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c60563-35ad-4080-98f8-bd22557c2cb5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.842272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619f09b-54e7-4715-8d01-dc9f63dce24c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:11.853975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:11.879076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9580cd-93e1-4107-bdcb-90cc7e8c1aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:11.879743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9580cd-93e1-4107-bdcb-90cc7e8c1aab None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.880015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9580cd-93e1-4107-bdcb-90cc7e8c1aab None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.880278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9580cd-93e1-4107-bdcb-90cc7e8c1aab None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.881383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9580cd-93e1-4107-bdcb-90cc7e8c1aab None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:11.882831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9580cd-93e1-4107-bdcb-90cc7e8c1aab None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:11.884465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 234/445] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:11.885323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9580cd-93e1-4107-bdcb-90cc7e8c1aab None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.914931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c60563-35ad-4080-98f8-bd22557c2cb5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:11.955866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d085f-78de-4251-88e0-b6c0c1c6618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:11.956604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d085f-78de-4251-88e0-b6c0c1c6618f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:11.956873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d085f-78de-4251-88e0-b6c0c1c6618f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:11.957147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d085f-78de-4251-88e0-b6c0c1c6618f None admin] PATH_INFO: `/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:11.958381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d085f-78de-4251-88e0-b6c0c1c6618f None admin] RBAC: Authorizing `identity:get_project(project_id=bb8bbbfbab8e46d49858d9ef99a96374)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:11.959894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d085f-78de-4251-88e0-b6c0c1c6618f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:11.966770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 212/446] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/projects/bb8bbbfbab8e46d49858d9ef99a96374 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:11.966770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d085f-78de-4251-88e0-b6c0c1c6618f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:11.983922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:11.997623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9580cd-93e1-4107-bdcb-90cc7e8c1aab None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:12.035944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449e7d6-c9c2-4309-bcb7-460ba7dacbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:12.036489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e7d6-c9c2-4309-bcb7-460ba7dacbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:12.037242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e7d6-c9c2-4309-bcb7-460ba7dacbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:12.037242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e7d6-c9c2-4309-bcb7-460ba7dacbe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:12.048140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449e7d6-c9c2-4309-bcb7-460ba7dacbe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7449e7d6-c9c2-4309-bcb7-460ba7dacbe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:12.053025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:12.053951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:12.054749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 235/447] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:12.055427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449e7d6-c9c2-4309-bcb7-460ba7dacbe7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:12.080023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d085f-78de-4251-88e0-b6c0c1c6618f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:12.109322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:12.117282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:12.117903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:12.118178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:12.118446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:12.119173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] RBAC: Adding query filter params (group.id=289105f07754455c9316a1ee7452c1e4, role.id=07097b9e79ae4ff1bf60ddef2b681c51, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:12.119507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:12.120977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:12.127654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 213/448] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:11 2022] GET /identity/v3/role_assignments?group.id=289105f07754455c9316a1ee7452c1e4&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=07097b9e79ae4ff1bf60ddef2b681c51 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:12.128447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:12.185788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449e7d6-c9c2-4309-bcb7-460ba7dacbe7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:12.235620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b463bb4-0797-4f58-ba25-32f5449b182c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:12.237438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463bb4-0797-4f58-ba25-32f5449b182c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:12.237889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463bb4-0797-4f58-ba25-32f5449b182c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:12.238345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463bb4-0797-4f58-ba25-32f5449b182c None admin] PATH_INFO: `/v3/groups/289105f07754455c9316a1ee7452c1e4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:12.240372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b463bb4-0797-4f58-ba25-32f5449b182c None admin] RBAC: Authorizing `identity:get_group(group_id=289105f07754455c9316a1ee7452c1e4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:12.241136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:12.243105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b463bb4-0797-4f58-ba25-32f5449b182c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:12.253572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 236/449] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:12 2022] GET /identity/v3/groups/289105f07754455c9316a1ee7452c1e4 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:12.255334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b463bb4-0797-4f58-ba25-32f5449b182c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:12.289978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39f527-90b8-4024-89ea-a251a90f072e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:12.343912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:12.344545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:12.344831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:12.345097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:12.345771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:12.346067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:12.347508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:12.367154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 214/450] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:12.374157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:12.433564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b463bb4-0797-4f58-ba25-32f5449b182c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:12.445339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:12.490102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:12.490757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:12.491035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:12.491366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:12.492510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] RBAC: Adding query filter params (group.id=289105f07754455c9316a1ee7452c1e4, role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:12.492826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:12.494753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:12.503861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 237/451] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:12 2022] GET /identity/v3/role_assignments?group.id=289105f07754455c9316a1ee7452c1e4&scope.project.id=bb8bbbfbab8e46d49858d9ef99a96374&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 611 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:12.531961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1bf6-a274-4b89-8864-5b9a5aa6250c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:12.584041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adddfb3-8e63-40be-9a78-131ffcc57884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:12.584728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adddfb3-8e63-40be-9a78-131ffcc57884 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:12.585004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adddfb3-8e63-40be-9a78-131ffcc57884 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:12.585274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adddfb3-8e63-40be-9a78-131ffcc57884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:12.600550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:12.688076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adddfb3-8e63-40be-9a78-131ffcc57884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:12.689686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adddfb3-8e63-40be-9a78-131ffcc57884 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:12.824793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 215/452] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:12.830887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:12.947454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6cf86-328e-42b8-9a2f-1700dda1b9a9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:12.984288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda1272-eec3-431e-9d49-743cc7730edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:12.984915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1272-eec3-431e-9d49-743cc7730edb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:12.985186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1272-eec3-431e-9d49-743cc7730edb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:12.985451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1272-eec3-431e-9d49-743cc7730edb None admin] PATH_INFO: `/v3/projects/b6117aaa4eb34295b2216e51a40eacde` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:12.986692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda1272-eec3-431e-9d49-743cc7730edb None admin] RBAC: Authorizing `identity:get_project(project_id=b6117aaa4eb34295b2216e51a40eacde)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:12.988323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda1272-eec3-431e-9d49-743cc7730edb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:12.990026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 238/453] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:05:12 2022] GET /identity/v3/projects/b6117aaa4eb34295b2216e51a40eacde => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:12.995916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adddfb3-8e63-40be-9a78-131ffcc57884 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:13.042490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:13.105089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adddfb3-8e63-40be-9a78-131ffcc57884 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:13.140294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c1180-1d4e-4579-bae0-d787b84af1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:13.140926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1180-1d4e-4579-bae0-d787b84af1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:13.141196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1180-1d4e-4579-bae0-d787b84af1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:13.141461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1180-1d4e-4579-bae0-d787b84af1c7 None admin] PATH_INFO: `/v3/groups/6ab967463d8b458a89c6215291537c9f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:13.142616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c1180-1d4e-4579-bae0-d787b84af1c7 None admin] RBAC: Authorizing `identity:get_group(group_id=6ab967463d8b458a89c6215291537c9f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:13.144212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c1180-1d4e-4579-bae0-d787b84af1c7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:13.145879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 216/454] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 31 12:05:12 2022] GET /identity/v3/groups/6ab967463d8b458a89c6215291537c9f => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:13.152266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda1272-eec3-431e-9d49-743cc7730edb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:13.198097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:13.265222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda1272-eec3-431e-9d49-743cc7730edb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:13.301317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069f7c2-f923-4221-a1ec-d7488bcdd41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:13.301913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069f7c2-f923-4221-a1ec-d7488bcdd41f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:13.302200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069f7c2-f923-4221-a1ec-d7488bcdd41f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:13.302466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069f7c2-f923-4221-a1ec-d7488bcdd41f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:13.303189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069f7c2-f923-4221-a1ec-d7488bcdd41f None admin] RBAC: Adding query filter params (group.id=6ab967463d8b458a89c6215291537c9f, role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=b6117aaa4eb34295b2216e51a40eacde) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:13.303532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069f7c2-f923-4221-a1ec-d7488bcdd41f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:13.304965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069f7c2-f923-4221-a1ec-d7488bcdd41f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:13.311017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 239/455] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 31 12:05:13 2022] GET /identity/v3/role_assignments?group.id=6ab967463d8b458a89c6215291537c9f&scope.project.id=b6117aaa4eb34295b2216e51a40eacde&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:15.570774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34be37-f77e-4575-b25a-b6950e5b67f4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:15.571950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34be37-f77e-4575-b25a-b6950e5b67f4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:15.572444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34be37-f77e-4575-b25a-b6950e5b67f4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:15.574272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 217/456] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:15.579807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9734a6-b26c-4a0b-9459-f67323f7dd2c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:15.580253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9734a6-b26c-4a0b-9459-f67323f7dd2c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:15.580352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9734a6-b26c-4a0b-9459-f67323f7dd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:15.607975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fd9734a6-b26c-4a0b-9459-f67323f7dd2c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:15.610036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fd9734a6-b26c-4a0b-9459-f67323f7dd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:15.701447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 240/457] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:16.056489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37241-7d29-4243-a1e8-3bc7a0c59286 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.057182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37241-7d29-4243-a1e8-3bc7a0c59286 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.057657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37241-7d29-4243-a1e8-3bc7a0c59286 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.059457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 218/458] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:16.065569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66087-0d3d-4bd3-ad12-5ad320357116 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.066118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66087-0d3d-4bd3-ad12-5ad320357116 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.066582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66087-0d3d-4bd3-ad12-5ad320357116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.086081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-79c66087-0d3d-4bd3-ad12-5ad320357116 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:16.087605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-79c66087-0d3d-4bd3-ad12-5ad320357116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:16.105239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eee5a-cab6-4d66-a6f2-6d0d0eb58202 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.105834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eee5a-cab6-4d66-a6f2-6d0d0eb58202 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.106298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eee5a-cab6-4d66-a6f2-6d0d0eb58202 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.108173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 219/459] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:16.114080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d5ab-bfed-4530-a5e5-92ea72247d6f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.114898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d5ab-bfed-4530-a5e5-92ea72247d6f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.115666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d5ab-bfed-4530-a5e5-92ea72247d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.127265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:16.153194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a002d5ab-bfed-4530-a5e5-92ea72247d6f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:16.154295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a002d5ab-bfed-4530-a5e5-92ea72247d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:16.199749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 241/460] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:05:16.255006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 220/461] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:16.331393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c66087-0d3d-4bd3-ad12-5ad320357116 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:16.448998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c66087-0d3d-4bd3-ad12-5ad320357116 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:16.490655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:16.491322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.491606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.491881 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.502692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:16.504213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:16.523938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:16.524903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:16.525737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:16.525737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:16.525737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:16.525737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 242/462] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:16.532235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002d5ab-bfed-4530-a5e5-92ea72247d6f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:16.568645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:16.603615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:16.606642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:16.684363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002d5ab-bfed-4530-a5e5-92ea72247d6f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:16.721919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3020a-07c3-4da3-9b0a-6e4a1c1e4ec8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:16.730229 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:16.731447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.731734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.732011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.732667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:16.732999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:16.735540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:16.748921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 221/463] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:16.750161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:16.770071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:16.770946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.771258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.771555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.786965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:16.788510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:16.808102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:16.809079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:16.810019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:16.810019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:16.810019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:16.810019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 243/464] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:16.815613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:16.864307 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:16.869516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c8d2f-abdb-4669-a266-617cb6aa9241 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:16.898376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:16.916033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:16.916671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.916953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.917229 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.927971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:16.929462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:16.941545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a34c84-763d-4228-91b6-b516bca24b9d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:16.944431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:16.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:16.946177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:16.946177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:16.946177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:16.946177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 222/465] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:16.946396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:16.980883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acfd57-adba-41d5-9e11-fc5d14b2c257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:16.981918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acfd57-adba-41d5-9e11-fc5d14b2c257 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:16.982508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acfd57-adba-41d5-9e11-fc5d14b2c257 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:16.982852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acfd57-adba-41d5-9e11-fc5d14b2c257 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:16.983833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acfd57-adba-41d5-9e11-fc5d14b2c257 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:16.985187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acfd57-adba-41d5-9e11-fc5d14b2c257 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:17.009480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 244/466] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:05:16 2022] POST /identity/v3/users => generated 246 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:17.012030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2331c-ad63-4222-a59f-46c8914a06e7 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.012515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2331c-ad63-4222-a59f-46c8914a06e7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.012961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2331c-ad63-4222-a59f-46c8914a06e7 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.020832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 245/467] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:17.022191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2331c-ad63-4222-a59f-46c8914a06e7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:17.086802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e38b8-1915-4409-9942-d40ed9b875d0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:17.129826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd96e1-2647-45c3-a795-aefc1cc7caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:17.130775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd96e1-2647-45c3-a795-aefc1cc7caf9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.131298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd96e1-2647-45c3-a795-aefc1cc7caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.131775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd96e1-2647-45c3-a795-aefc1cc7caf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.132709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd96e1-2647-45c3-a795-aefc1cc7caf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:17.133189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd96e1-2647-45c3-a795-aefc1cc7caf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:17.134774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd96e1-2647-45c3-a795-aefc1cc7caf9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:17.149464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 223/468] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:17.151191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0244d-48cf-4005-915f-9e707147dcf5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.151654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0244d-48cf-4005-915f-9e707147dcf5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.152084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0244d-48cf-4005-915f-9e707147dcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.168924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2331c-ad63-4222-a59f-46c8914a06e7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:17.173104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fae0244d-48cf-4005-915f-9e707147dcf5 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:17.174309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fae0244d-48cf-4005-915f-9e707147dcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:17.177859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:17.198247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:17.214296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:17.215423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.215423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.215615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.216138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:17.217037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:17.217868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:17.231041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 246/469] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:17.232104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:17.266230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 224/470] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:17.267335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0244d-48cf-4005-915f-9e707147dcf5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:17.370554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8089b2e-f389-47db-962e-b4ade17f33fa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:17.390363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0244d-48cf-4005-915f-9e707147dcf5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:17.407514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16f8a8-0357-4f4b-9cf6-ba65e46cdbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:17.408371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f8a8-0357-4f4b-9cf6-ba65e46cdbb8 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.408804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f8a8-0357-4f4b-9cf6-ba65e46cdbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.409217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f8a8-0357-4f4b-9cf6-ba65e46cdbb8 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.410018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16f8a8-0357-4f4b-9cf6-ba65e46cdbb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:17.411688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16f8a8-0357-4f4b-9cf6-ba65e46cdbb8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:17.435989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0e5b-a481-4ee6-97ec-ee75dcd9c155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:17.437452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0e5b-a481-4ee6-97ec-ee75dcd9c155 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.438222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0e5b-a481-4ee6-97ec-ee75dcd9c155 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.438671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0e5b-a481-4ee6-97ec-ee75dcd9c155 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.440104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf0e5b-a481-4ee6-97ec-ee75dcd9c155 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:17.442095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf0e5b-a481-4ee6-97ec-ee75dcd9c155 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:17.454405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 247/471] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:05:17 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:17.456031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a44ea4-c304-44b6-a989-410e19bc6408 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.456426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a44ea4-c304-44b6-a989-410e19bc6408 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.456801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a44ea4-c304-44b6-a989-410e19bc6408 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.457167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:17.458793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 248/472] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:17.464664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490c605-7838-4c7c-b19c-2de28f6a9a76 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.464955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490c605-7838-4c7c-b19c-2de28f6a9a76 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.465338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490c605-7838-4c7c-b19c-2de28f6a9a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.478337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 225/473] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:05:17 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:17.490090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d490c605-7838-4c7c-b19c-2de28f6a9a76 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:17.491495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d490c605-7838-4c7c-b19c-2de28f6a9a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:17.528587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:17.573409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 249/474] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:17.577783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0e5b-a481-4ee6-97ec-ee75dcd9c155 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:17.694083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0e5b-a481-4ee6-97ec-ee75dcd9c155 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:17.743680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:17.744260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.744540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.744811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.760855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:17.762334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:17.779637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:17.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:17.781971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:17.781971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:17.781971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:17.781971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 226/475] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:17.788223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490c605-7838-4c7c-b19c-2de28f6a9a76 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:17.882821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:17.906977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490c605-7838-4c7c-b19c-2de28f6a9a76 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:17.947501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:17.948526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:17.949080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:17.949934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:17.950425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:17.951307 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:17.951884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:17.954319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:17.968033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 250/476] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:17.975367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:18.006731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639583d-5766-4444-910d-e6df5f8533e0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:18.041578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:18.052632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:18.053390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:18.053523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:18.054736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:18.054736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:18.056223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:18.075514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.sql.core [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 12:05:18.075514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 12:05:18.075514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [parameters: {'id': '55ef38badc044c17abb61362d40a468c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 12:05:18.075514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81487) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application conn.query(q) Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result.read() Aug 31 12:05:18.100168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 12:05:18.101184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application next(self.gen) Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application yield resource Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application next(self.gen) Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise self.value Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application session.commit() Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 12:05:18.102106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.session.flush() Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application compat.raise_( Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 12:05:18.102951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 12:05:18.103865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application conn.query(q) Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result.read() Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 12:05:18.104684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [parameters: {'id': '55ef38badc044c17abb61362d40a468c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:18.105496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 12:05:18.106308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:18.107154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 227/477] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 31 12:05:17 2022] POST /identity/v3/roles => generated 144 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:18.113999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:18.235620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2374d5-b11b-45ab-8403-0464e2798a62 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:18.246011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24db5f6-6b32-4e5e-a1b3-75aa74eeb7e5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:18.264155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:18.277570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702fed6-8b86-43ee-9272-6f2ecba730a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:18.278263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702fed6-8b86-43ee-9272-6f2ecba730a0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:18.278548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702fed6-8b86-43ee-9272-6f2ecba730a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:18.279221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702fed6-8b86-43ee-9272-6f2ecba730a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:18.292198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702fed6-8b86-43ee-9272-6f2ecba730a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d702fed6-8b86-43ee-9272-6f2ecba730a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:18.297818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:18.299900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:18.301820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 228/478] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:18.305894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702fed6-8b86-43ee-9272-6f2ecba730a0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:18.361520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640d801-e273-440c-854e-f12eea9923e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:18.362146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640d801-e273-440c-854e-f12eea9923e1 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:18.362420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640d801-e273-440c-854e-f12eea9923e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:18.362685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640d801-e273-440c-854e-f12eea9923e1 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:18.363397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640d801-e273-440c-854e-f12eea9923e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:18.364855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640d801-e273-440c-854e-f12eea9923e1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:18.410037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 251/479] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:05:17 2022] POST /identity/v3/users => generated 249 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:18.427528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702fed6-8b86-43ee-9272-6f2ecba730a0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:18.467529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0d6c2-e8b1-4cfc-84f4-723904011ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:18.468475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0d6c2-e8b1-4cfc-84f4-723904011ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:18.468770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0d6c2-e8b1-4cfc-84f4-723904011ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:18.469045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0d6c2-e8b1-4cfc-84f4-723904011ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:18.469733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0d6c2-e8b1-4cfc-84f4-723904011ec7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:18.470040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0d6c2-e8b1-4cfc-84f4-723904011ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:18.471473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0d6c2-e8b1-4cfc-84f4-723904011ec7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:18.481713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 229/480] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:19.455526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ee9ed-b842-4e2d-84c4-399a22b2df0c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:19.456087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ee9ed-b842-4e2d-84c4-399a22b2df0c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:19.456087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ee9ed-b842-4e2d-84c4-399a22b2df0c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:19.457561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 252/481] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:19.462672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12cd2f-ed7d-40e1-ae01-e863ec6dd525 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:19.463091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12cd2f-ed7d-40e1-ae01-e863ec6dd525 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:19.463199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12cd2f-ed7d-40e1-ae01-e863ec6dd525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:19.488248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dd12cd2f-ed7d-40e1-ae01-e863ec6dd525 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:19.489444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dd12cd2f-ed7d-40e1-ae01-e863ec6dd525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:19.521799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:19.582132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 230/482] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:19.978440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ee9ed-b842-4e2d-84c4-399a22b2df0c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:19.988749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b828a-8a42-4027-82f7-4e136f3cd7aa None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:19.989342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b828a-8a42-4027-82f7-4e136f3cd7aa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:19.989811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b828a-8a42-4027-82f7-4e136f3cd7aa None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:19.991619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 231/483] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:19.997339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e2d9-58b1-4474-8860-33a4be56f5ad None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:19.997904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e2d9-58b1-4474-8860-33a4be56f5ad None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:19.998376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e2d9-58b1-4474-8860-33a4be56f5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.017607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9727e2d9-58b1-4474-8860-33a4be56f5ad None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:20.019160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9727e2d9-58b1-4474-8860-33a4be56f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:20.085255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:20.097557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 232/484] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:05:20.104707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070a8c1-3c05-4eff-a64d-998493dc3ca5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.104903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ee9ed-b842-4e2d-84c4-399a22b2df0c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:20.105374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070a8c1-3c05-4eff-a64d-998493dc3ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.105842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070a8c1-3c05-4eff-a64d-998493dc3ca5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.111944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 233/485] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:20 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:20.116951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028e3b1-29c7-48e7-ac6a-bbaaf56d6c6d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.117451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028e3b1-29c7-48e7-ac6a-bbaaf56d6c6d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.117919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028e3b1-29c7-48e7-ac6a-bbaaf56d6c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.146029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a494954-f389-4054-ad27-57e3f0e70e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:20.146699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a494954-f389-4054-ad27-57e3f0e70e83 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.146980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a494954-f389-4054-ad27-57e3f0e70e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.147263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a494954-f389-4054-ad27-57e3f0e70e83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.148617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9028e3b1-29c7-48e7-ac6a-bbaaf56d6c6d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:20.149792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9028e3b1-29c7-48e7-ac6a-bbaaf56d6c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:20.170482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a494954-f389-4054-ad27-57e3f0e70e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9a494954-f389-4054-ad27-57e3f0e70e83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:20.175262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:20.176141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:20.176943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 253/486] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:20.184592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a494954-f389-4054-ad27-57e3f0e70e83 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:20.281971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 234/487] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:05:20.303179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a494954-f389-4054-ad27-57e3f0e70e83 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:20.336714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:20.337319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.337593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.337862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.338806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:20.339111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:20.340516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:20.349440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 254/488] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:20.354801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028e3b1-29c7-48e7-ac6a-bbaaf56d6c6d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:20.401790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:20.519517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028e3b1-29c7-48e7-ac6a-bbaaf56d6c6d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:20.568720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:20.569731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.570231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.570691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.581702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:20.583489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:20.588097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:20.599279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:20.605068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:20.605978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:20.607295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:20.607295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:20.607295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:20.607295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 235/489] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:20.612179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:20.708673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:20.714787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd93e0-d9a3-4a19-9f4f-646a83bdd1b8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:20.744600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cae26-420d-45ea-8e7a-1bc200f5c40c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:20.752466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72b3bb-dd87-45f5-ad19-f547adee6184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:20.753101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b3bb-dd87-45f5-ad19-f547adee6184 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.753391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b3bb-dd87-45f5-ad19-f547adee6184 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.753649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b3bb-dd87-45f5-ad19-f547adee6184 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.765584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72b3bb-dd87-45f5-ad19-f547adee6184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1b72b3bb-dd87-45f5-ad19-f547adee6184 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:20.771337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:20.772252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 255/490] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:20.773412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72b3bb-dd87-45f5-ad19-f547adee6184 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:20.791796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:20.792745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.793191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.793833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.794983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:20.795556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:20.797126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:20.814372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 236/491] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:20.815594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:20.854504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:20.896633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72b3bb-dd87-45f5-ad19-f547adee6184 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:20.933332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8198c4-0bd1-4ff6-9100-d3f68d5724b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:20.933942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8198c4-0bd1-4ff6-9100-d3f68d5724b7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.934208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8198c4-0bd1-4ff6-9100-d3f68d5724b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.934479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8198c4-0bd1-4ff6-9100-d3f68d5724b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.940557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa9a50-9f42-42d0-9648-7a41125fd187 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:20.947802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8198c4-0bd1-4ff6-9100-d3f68d5724b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cb8198c4-0bd1-4ff6-9100-d3f68d5724b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:20.954320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:20.955195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:20.956059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 256/492] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:20.956731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8198c4-0bd1-4ff6-9100-d3f68d5724b7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:20.980881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fd2f0-1572-476e-b9a6-b716a82e2d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:20.981632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fd2f0-1572-476e-b9a6-b716a82e2d7b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.981933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fd2f0-1572-476e-b9a6-b716a82e2d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.982210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fd2f0-1572-476e-b9a6-b716a82e2d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.982928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fd2f0-1572-476e-b9a6-b716a82e2d7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:20.983261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fd2f0-1572-476e-b9a6-b716a82e2d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:20.984645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fd2f0-1572-476e-b9a6-b716a82e2d7b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:20.986811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:20.993250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 237/493] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:20.994505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d7367-4e35-4874-a810-14a8b8a9ec2f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:20.994791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d7367-4e35-4874-a810-14a8b8a9ec2f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:20.995062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d7367-4e35-4874-a810-14a8b8a9ec2f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:20.999274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 238/494] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:20.999274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d7367-4e35-4874-a810-14a8b8a9ec2f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.058329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8198c4-0bd1-4ff6-9100-d3f68d5724b7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.082695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:21.091514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fe160-09fb-4020-a9a4-f73c9d09a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.092166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fe160-09fb-4020-a9a4-f73c9d09a9ba None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.092443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fe160-09fb-4020-a9a4-f73c9d09a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.092719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fe160-09fb-4020-a9a4-f73c9d09a9ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.096378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d7367-4e35-4874-a810-14a8b8a9ec2f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.100469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fe160-09fb-4020-a9a4-f73c9d09a9ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:21.101924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fe160-09fb-4020-a9a4-f73c9d09a9ba None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b36fe160-09fb-4020-a9a4-f73c9d09a9ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:21.114207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:21.115258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:21.116142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 257/495] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:05:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:21.120463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceae0d8-aeb4-4019-ae61-ab420b8acb53 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.120739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceae0d8-aeb4-4019-ae61-ab420b8acb53 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.121010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceae0d8-aeb4-4019-ae61-ab420b8acb53 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.123276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 258/496] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:21.123276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceae0d8-aeb4-4019-ae61-ab420b8acb53 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.133324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909b081-ff66-49c9-9744-ce564b9cd7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.134062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909b081-ff66-49c9-9744-ce564b9cd7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.134433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909b081-ff66-49c9-9744-ce564b9cd7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.134795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909b081-ff66-49c9-9744-ce564b9cd7b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.135542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909b081-ff66-49c9-9744-ce564b9cd7b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:21.135936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909b081-ff66-49c9-9744-ce564b9cd7b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:21.137645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909b081-ff66-49c9-9744-ce564b9cd7b6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:21.148167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 239/497] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:21.149457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a0faf-6402-44cb-aff1-fa9d2bcd27f7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.149722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a0faf-6402-44cb-aff1-fa9d2bcd27f7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.150014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a0faf-6402-44cb-aff1-fa9d2bcd27f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.168321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-541a0faf-6402-44cb-aff1-fa9d2bcd27f7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:21.169536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-541a0faf-6402-44cb-aff1-fa9d2bcd27f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:21.225414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:21.231083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceae0d8-aeb4-4019-ae61-ab420b8acb53 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.237801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 240/498] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:21.238580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a0faf-6402-44cb-aff1-fa9d2bcd27f7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.276655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad7159-a92e-45d8-b20a-010caeb4d2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.277273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad7159-a92e-45d8-b20a-010caeb4d2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.277544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad7159-a92e-45d8-b20a-010caeb4d2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.277808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad7159-a92e-45d8-b20a-010caeb4d2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.289010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad7159-a92e-45d8-b20a-010caeb4d2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:21.290595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad7159-a92e-45d8-b20a-010caeb4d2b5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-81ad7159-a92e-45d8-b20a-010caeb4d2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:21.305468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:21.306392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:21.307255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:21.307255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:21.307255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:21.311830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 259/499] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:21.312950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800431a6-3f13-4cae-91ff-91aefa198519 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.313233 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800431a6-3f13-4cae-91ff-91aefa198519 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.313502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800431a6-3f13-4cae-91ff-91aefa198519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.338806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-800431a6-3f13-4cae-91ff-91aefa198519 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:21.340051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-800431a6-3f13-4cae-91ff-91aefa198519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:21.357440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a0faf-6402-44cb-aff1-fa9d2bcd27f7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.373136 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:21.395874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.396478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.396887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.397021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.397708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:21.398005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:21.399474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:21.419266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 260/500] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:21.419266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800431a6-3f13-4cae-91ff-91aefa198519 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.464770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 241/501] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:21.465667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.519289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800431a6-3f13-4cae-91ff-91aefa198519 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.525904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:21.556100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.556686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.556962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.557230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.567955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:21.569421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:21.583422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd4150-0a9a-4a39-96d1-5bd2ea2403e9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:21.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:21.587298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:21.588138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:21.588138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:21.588138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:21.588138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 261/502] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:21.588138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.611521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:21.624608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.625407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.625842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.626099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.626737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:21.627032 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:21.628461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:21.644744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 242/503] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:21.644744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.698940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4bee0-d3c1-407e-9b37-275cfe19a33e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.731974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deb2bd-3e3d-4ad4-9893-08038b661c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.732575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deb2bd-3e3d-4ad4-9893-08038b661c82 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.732906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deb2bd-3e3d-4ad4-9893-08038b661c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.733176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deb2bd-3e3d-4ad4-9893-08038b661c82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.745749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39693d18-cf3a-42e9-9e48-eb9e301269d7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.756506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:21.784015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae606c-9372-441b-aa20-5c2b62317ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.784571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae606c-9372-441b-aa20-5c2b62317ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.784844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae606c-9372-441b-aa20-5c2b62317ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.785129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae606c-9372-441b-aa20-5c2b62317ba5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.795104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae606c-9372-441b-aa20-5c2b62317ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fbae606c-9372-441b-aa20-5c2b62317ba5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:21.800288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:21.801275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:21.802051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 243/504] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:21.802708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae606c-9372-441b-aa20-5c2b62317ba5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.804267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deb2bd-3e3d-4ad4-9893-08038b661c82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:21.806130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deb2bd-3e3d-4ad4-9893-08038b661c82 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:21.827186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:21.905149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 262/505] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:21.906146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deb2bd-3e3d-4ad4-9893-08038b661c82 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.907087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae606c-9372-441b-aa20-5c2b62317ba5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:21.940927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:21.941535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:21.941814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:21.942096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:21.942728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:21.943033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:21.944477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:21.960937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 244/506] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:21.962036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:21.990595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.011157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deb2bd-3e3d-4ad4-9893-08038b661c82 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.031055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.045865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.046497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.046772 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.047052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.054069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.055559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.061745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9aaba-449b-4b9e-bbb7-fa1e141a2861 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:22.067137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:22.068062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:22.068947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 263/507] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:22.069509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.096184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.096818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.097096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.097368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.107529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.109250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:22.123457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:22.124522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:22.125457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:22.125457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:22.125457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:22.125457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 245/508] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:22.125457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.176822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab507bc-4532-40ce-ac38-d5ea545d16bf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.207471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.210435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.211035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.211344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.211616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.212266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:22.212570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.213939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.227087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 264/509] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:22.227919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.228372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45510d6d-9de8-46fe-bed8-8e106a377e01 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.266556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.267185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.267498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.267771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.274269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.278835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.280416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:22.294462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:22.295734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:22.296834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:22.296834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:22.296834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:22.296834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 246/510] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:22.296834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.339005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754753cd-a828-4c3a-ad23-f6539829c2e4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.373564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.374304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.374543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.374833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.382323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.383810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:22.394951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:22.396062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 265/511] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:22.396934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.416696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80fcd-f3ca-409c-a9c2-6c562b0840af None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.445736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.451773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134a971-2b21-4015-8740-494c6370827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.452375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134a971-2b21-4015-8740-494c6370827e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.452650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134a971-2b21-4015-8740-494c6370827e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.452922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134a971-2b21-4015-8740-494c6370827e None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.453643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134a971-2b21-4015-8740-494c6370827e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:22.453935 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134a971-2b21-4015-8740-494c6370827e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.455411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134a971-2b21-4015-8740-494c6370827e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.473249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 247/512] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:22.474357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134a971-2b21-4015-8740-494c6370827e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.498530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9c8ee-ca12-436b-ac4a-d202f428a8a7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.500100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.556999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.557773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.558193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.558469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.559137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:22.559471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.560944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.574153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 266/513] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:22.574997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.583368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134a971-2b21-4015-8740-494c6370827e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.618954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.619630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.619906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.620176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.630333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.631837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:22.647090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:22.648231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:22.649081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:22.649081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:22.649081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:22.649081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 248/514] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:22.649081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.680777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da313b47-925e-4395-b809-e97a7b66d991 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.696040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.714657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.715299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.715583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.715858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.716499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:22.716803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.718270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.730200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 267/515] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:22.731077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.753971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8de6b-fae4-4f05-97d6-41e8752f350a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.760152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.788155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.788758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.789034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.789303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.790016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:22.790318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.791794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.815750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 249/516] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:22.816291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:22.847021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e132d7-e88e-4f99-9e60-240e2f9e978c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.881323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af844533-5ae7-44d4-83ba-5d0c43bf3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.881931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af844533-5ae7-44d4-83ba-5d0c43bf3754 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.882204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af844533-5ae7-44d4-83ba-5d0c43bf3754 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.882448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af844533-5ae7-44d4-83ba-5d0c43bf3754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.914099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.917197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e6074-be7f-4de4-96da-2c9fdcdf5635 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:22.949558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af844533-5ae7-44d4-83ba-5d0c43bf3754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.951406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af844533-5ae7-44d4-83ba-5d0c43bf3754 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.954437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9db7b8-d838-4f2e-992a-94390b74c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:22.955058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db7b8-d838-4f2e-992a-94390b74c863 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:22.955360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db7b8-d838-4f2e-992a-94390b74c863 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:22.955636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db7b8-d838-4f2e-992a-94390b74c863 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:22.956319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9db7b8-d838-4f2e-992a-94390b74c863 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:22.957719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9db7b8-d838-4f2e-992a-94390b74c863 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:22.975251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:22.979819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 250/517] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:05:22 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:22.980627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9db7b8-d838-4f2e-992a-94390b74c863 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.063928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 268/518] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:23.063928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af844533-5ae7-44d4-83ba-5d0c43bf3754 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.121980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9db7b8-d838-4f2e-992a-94390b74c863 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.168800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.169689 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.170264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.170689 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.171512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:23.171954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.173493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.176216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af844533-5ae7-44d4-83ba-5d0c43bf3754 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.187904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 251/519] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:23.188752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.201633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:23.211531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:23.215116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.215907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.216280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.216656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.224930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.226400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:23.238712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:23.239642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:23.240483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 269/520] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:23.241648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.307983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169ee76-4acd-4ea0-b816-7ed0c26029e7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.342075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e85e34-8dd6-4a3a-b4f0-edc9670783b4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.349062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e9829-7d51-416e-802b-c7471f5f8e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.350101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e9829-7d51-416e-802b-c7471f5f8e71 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.350392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e9829-7d51-416e-802b-c7471f5f8e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.350683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e9829-7d51-416e-802b-c7471f5f8e71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.378058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.378610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.378886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.379167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.390256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.391749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:23.407978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:23.408929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:23.409826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:23.409826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:23.409826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:23.409826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 270/521] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:23.409826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.426252 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e9829-7d51-416e-802b-c7471f5f8e71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.427769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e9829-7d51-416e-802b-c7471f5f8e71 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.446568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:23.447687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:23.519215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca9065-9e0d-44ff-8ec8-bad579636577 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.544730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 252/522] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:23.544730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e9829-7d51-416e-802b-c7471f5f8e71 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.566889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.567562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.567842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.568112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.576948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.578432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:23.590082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:23.590971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 271/523] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:23.592296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.658787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e9829-7d51-416e-802b-c7471f5f8e71 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.678891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:23.693310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:23.699902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.700519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.700792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.700907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5513f1-c9be-468b-a764-6bea3c6e6104 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.701058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.701865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:23.702166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.703677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.721455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 253/524] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:23.722268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.745666 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.746454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.746739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.747343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.747735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:23.748104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.749727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.769371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 272/525] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:23.769371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.838260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a054f-2989-4552-8d4a-6a556f257ca7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.892481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.892481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.892481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.892850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f83db0-bcd0-48d2-accf-247647b9ae40 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:23.892935 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.905989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.907552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:23.923938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:23.926865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:23.929193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:23.929193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:23.929193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:23.929193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 254/526] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:23.929193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:23.944461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:23.945011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:23.945290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:23.945559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:23.947200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:23.947200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:23.948103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:23.978884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:23.993377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 273/527] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:23.994150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.009756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:24.065227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328748f3-0862-4ed2-ac49-f5540ef1c57f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:24.101923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:24.102555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:24.102824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:24.103085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:24.103910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=0b113316bc3348b5a12c1d7b616bf4c9) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:24.104218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:24.105733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:24.123268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 255/528] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/role_assignments?user.id=0b113316bc3348b5a12c1d7b616bf4c9&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:24.123268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.131109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2d55e-faaf-4119-9f87-f30bb6bde1e7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:24.178292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:24.179346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:24.179628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:24.179897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:24.190251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:24.192497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:24.210100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:24.211029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:24.211895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:24.211895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:24.211895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:24.211895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:24.211895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:24.211895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:24.211895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:24.211895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:24.212312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 274/529] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:24.217842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.238395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:24.250497 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc38b7-1ea5-43d1-a5e0-7c558e2aaf30 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:24.287798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:24.288417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:24.288694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:24.288963 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:24.289608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:24.289911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:24.291342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:24.307854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 256/530] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:24.308723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.309311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:24.346811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b1a27-d177-4957-9bce-b78e25311aaf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:24.393353 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6996768-64fe-42bd-9e84-88157d89ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:24.394315 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6996768-64fe-42bd-9e84-88157d89ed59 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:24.394785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6996768-64fe-42bd-9e84-88157d89ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:24.395306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6996768-64fe-42bd-9e84-88157d89ed59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:24.428158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba188bac-7b64-4471-aac7-251060f22911 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:24.472884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:24.474045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:24.474518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:24.474969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:24.475872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:24.476377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:24.478001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:24.487586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6996768-64fe-42bd-9e84-88157d89ed59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:24.489425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6996768-64fe-42bd-9e84-88157d89ed59 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:24.491280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 257/531] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:24.492336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.508361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:24.597022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:24.606895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a85583-bbcf-46e1-a4b4-228bd5c60b72 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:24.635945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 275/532] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:24.636947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6996768-64fe-42bd-9e84-88157d89ed59 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.643193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:24.644183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:24.644650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:24.645118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:24.654221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:24.656008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:24.668291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:24.669582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 258/533] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:24.676051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.742977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:24.767681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6996768-64fe-42bd-9e84-88157d89ed59 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:24.775356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312f5e7-a165-45dd-a062-003d2798c374 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:24.807670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:24.808640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:24.809107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:24.809606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:24.810517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:24.810636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=db3393d3cb0e4bbcb21e403c431bf61b) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:24.811156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:24.811288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:24.811465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:24.811734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:24.813347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:24.819784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 276/534] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:24 2022] GET /identity/v3/role_assignments?user.id=db3393d3cb0e4bbcb21e403c431bf61b&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:24.821066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.823161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:24.824696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:24.841989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:24.843941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:24.844802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:24.844802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:24.844802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:24.844802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 259/535] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:24.847712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:24.906016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:24.968322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e06855-cd8e-45fe-bc9a-6ba2fdd855e8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:25.007757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:25.017639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb474c6e-3da9-44ba-9843-5df6b4d62ead None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:25.022448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb078cc-9e4d-413a-936a-699cfba4a9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:25.023065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb078cc-9e4d-413a-936a-699cfba4a9ca None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.023419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb078cc-9e4d-413a-936a-699cfba4a9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.023687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb078cc-9e4d-413a-936a-699cfba4a9ca None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.024351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb078cc-9e4d-413a-936a-699cfba4a9ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:25.024651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb078cc-9e4d-413a-936a-699cfba4a9ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:25.026055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb078cc-9e4d-413a-936a-699cfba4a9ca None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:25.041353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 260/536] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:25.042863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0269-4500-43d0-b67f-48a1357d54c3 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.043143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0269-4500-43d0-b67f-48a1357d54c3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.043449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c0269-4500-43d0-b67f-48a1357d54c3 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.049903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 261/537] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:25.049903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c0269-4500-43d0-b67f-48a1357d54c3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:25.066827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02583c-6d4c-48c3-949c-c7f18c80713f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:25.067572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02583c-6d4c-48c3-949c-c7f18c80713f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.067810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02583c-6d4c-48c3-949c-c7f18c80713f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.068085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02583c-6d4c-48c3-949c-c7f18c80713f None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.068782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02583c-6d4c-48c3-949c-c7f18c80713f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:25.069085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02583c-6d4c-48c3-949c-c7f18c80713f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:25.070517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02583c-6d4c-48c3-949c-c7f18c80713f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:25.094508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 277/538] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:25.095538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921853f-9351-4ad6-a562-f0d4b63fcf61 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.095825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921853f-9351-4ad6-a562-f0d4b63fcf61 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.096098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921853f-9351-4ad6-a562-f0d4b63fcf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.116281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0921853f-9351-4ad6-a562-f0d4b63fcf61 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:25.117601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0921853f-9351-4ad6-a562-f0d4b63fcf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:25.170756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c0269-4500-43d0-b67f-48a1357d54c3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:25.191266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 278/539] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:25.191266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921853f-9351-4ad6-a562-f0d4b63fcf61 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:25.216596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:25.225130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:25.231500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.232066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.232563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.247272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:25.249494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:25.270350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:25.271251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:25.272089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:25.272089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:25.272089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:25.272089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:25.272089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:25.272089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:25.272089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:25.272089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:25.274467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 262/540] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:25.279911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:25.295304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921853f-9351-4ad6-a562-f0d4b63fcf61 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:25.329025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:25.329670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.329945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.330213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.330941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=eef756349e5844e59d7b20072f63c41c) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:25.331258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:25.332714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:25.343403 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 279/541] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:25 2022] GET /identity/v3/role_assignments?user.id=eef756349e5844e59d7b20072f63c41c&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:25.406793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:25.492560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72901c56-f96c-4929-a69f-dbdef8b25cc9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:25.529575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:25.530121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.530397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.530663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.531379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:25.531681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:25.533117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:25.545945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 263/542] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:25.556415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:25.571647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:25.723850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:25.740872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a68c3-93ac-43b2-a8c2-73b26b4bceaf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:25.770541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5efb5d-81fb-421b-9934-3c1930b4fac3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:25.774663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:25.812544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:25.813170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.813389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.813664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.814391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=ba8c7348ad2e457e80e1fdd2d93aacc6) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:25.814699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:25.816179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:25.822774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 280/543] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:25 2022] GET /identity/v3/role_assignments?user.id=ba8c7348ad2e457e80e1fdd2d93aacc6&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:25.832026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7375d-b69d-4054-b697-7f6f45430820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:25.833055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7375d-b69d-4054-b697-7f6f45430820 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:25.833536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7375d-b69d-4054-b697-7f6f45430820 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:25.834003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7375d-b69d-4054-b697-7f6f45430820 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:25.846692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7375d-b69d-4054-b697-7f6f45430820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2df7375d-b69d-4054-b697-7f6f45430820 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:25.852257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:25.853125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:25.853931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:25.856568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 264/544] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:25.862870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:25.990507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1465cfc-c4b7-4919-b463-70155fa33837 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:26.036031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:26.036674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:26.036953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:26.037224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:26.037910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:26.038210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:26.039723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:26.050555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 281/545] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:26.060392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7375d-b69d-4054-b697-7f6f45430820 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:26.165573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7375d-b69d-4054-b697-7f6f45430820 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:26.201091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:26.202157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:26.202654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:26.203163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:26.214009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:26.215786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:26.231332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:26.232248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:26.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:26.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:26.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:26.239272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 265/546] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:26.241152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:26.260722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:26.374726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca6cdd-adda-4d04-a817-e9e57774a6d9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:26.416559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:26.417123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:26.417402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:26.417728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:26.418378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:26.418681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:26.420154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:26.436524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 282/547] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:26.442554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:26.463954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:26.549291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ca42b-845c-4a0a-87af-8bc5892b475a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:26.584691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58ab82-c5f9-47da-bcc7-fbc69d66bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:26.585329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58ab82-c5f9-47da-bcc7-fbc69d66bce6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:26.585609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58ab82-c5f9-47da-bcc7-fbc69d66bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:26.585882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58ab82-c5f9-47da-bcc7-fbc69d66bce6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:26.594361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58ab82-c5f9-47da-bcc7-fbc69d66bce6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:26.595928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58ab82-c5f9-47da-bcc7-fbc69d66bce6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1b58ab82-c5f9-47da-bcc7-fbc69d66bce6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:26.609972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:26.610909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:26.613056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:26.615526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 266/548] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:26.617160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:26.641508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60cddd-4c8c-4ddc-80bc-22b5a68d311e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:26.641849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60cddd-4c8c-4ddc-80bc-22b5a68d311e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:26.642118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60cddd-4c8c-4ddc-80bc-22b5a68d311e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:26.643581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 267/549] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:05:26.648999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0b2cf-6f2b-4d9a-bfef-15ee84089194 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:26.649286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0b2cf-6f2b-4d9a-bfef-15ee84089194 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:26.649559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0b2cf-6f2b-4d9a-bfef-15ee84089194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:26.679738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-17b0b2cf-6f2b-4d9a-bfef-15ee84089194 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:26.680891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-17b0b2cf-6f2b-4d9a-bfef-15ee84089194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:26.739874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:26.764577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 268/550] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:05:26.777386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723872c-b622-406b-817e-0b6e3ea25f82 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:26.814094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:26.814844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:26.815036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:26.815345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:26.816087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:26.816393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:26.817951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:26.836206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 283/551] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 31 12:05:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:26.842066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0b2cf-6f2b-4d9a-bfef-15ee84089194 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:26.851574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:26.949339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0b2cf-6f2b-4d9a-bfef-15ee84089194 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:26.991343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e48f1d-df3b-4b11-a169-b6a0eac2f8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:26.991962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e48f1d-df3b-4b11-a169-b6a0eac2f8ac None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:26.992242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e48f1d-df3b-4b11-a169-b6a0eac2f8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:26.992517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e48f1d-df3b-4b11-a169-b6a0eac2f8ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:27.062224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e48f1d-df3b-4b11-a169-b6a0eac2f8ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:27.063782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e48f1d-df3b-4b11-a169-b6a0eac2f8ac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:27.075725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:27.084099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:27.179569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 269/552] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:27.180862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd343d-fb6f-4e34-b4e3-daa4ea2c4b46 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:27.181149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd343d-fb6f-4e34-b4e3-daa4ea2c4b46 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:27.181417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd343d-fb6f-4e34-b4e3-daa4ea2c4b46 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:27.191273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 270/553] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:27.191273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd343d-fb6f-4e34-b4e3-daa4ea2c4b46 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:27.275495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41457f1c-3d2a-442f-a248-40f772f7ddc5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:27.310427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd343d-fb6f-4e34-b4e3-daa4ea2c4b46 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:27.319817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:27.336847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68ff95-38c2-4822-8347-892fa16aed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:27.338323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68ff95-38c2-4822-8347-892fa16aed39 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:27.339083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68ff95-38c2-4822-8347-892fa16aed39 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:27.339863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68ff95-38c2-4822-8347-892fa16aed39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:27.343586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:27.351797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc96d4-d6a3-49ef-a198-9318cdcb3ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:27.352550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc96d4-d6a3-49ef-a198-9318cdcb3ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:27.352810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc96d4-d6a3-49ef-a198-9318cdcb3ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:27.353094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc96d4-d6a3-49ef-a198-9318cdcb3ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:27.363179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68ff95-38c2-4822-8347-892fa16aed39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:27.364831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc96d4-d6a3-49ef-a198-9318cdcb3ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:27.366325 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc96d4-d6a3-49ef-a198-9318cdcb3ee4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fe68ff95-38c2-4822-8347-892fa16aed39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:27.373705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:27.375424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:27.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-6edc96d4-d6a3-49ef-a198-9318cdcb3ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:27.385096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:27.386720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:27.392339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:27.392339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:27.392339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:27.392339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 271/555] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:27.392339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305e2f-96b1-4d74-b59d-dbc29aad06be None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:27.392339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305e2f-96b1-4d74-b59d-dbc29aad06be None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:27.392339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305e2f-96b1-4d74-b59d-dbc29aad06be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:27.393458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 284/555] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:27.395335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68ff95-38c2-4822-8347-892fa16aed39 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:27.410608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c3305e2f-96b1-4d74-b59d-dbc29aad06be None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:27.411844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c3305e2f-96b1-4d74-b59d-dbc29aad06be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:27.503015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 272/556] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:27.503015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305e2f-96b1-4d74-b59d-dbc29aad06be None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:27.594660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:27.609579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305e2f-96b1-4d74-b59d-dbc29aad06be None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:27.642922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68ff95-38c2-4822-8347-892fa16aed39 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:27.647944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:27.648565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:27.648851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:27.649124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:27.649795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:27.650099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:27.655073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:27.662115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 273/557] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:27.671260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:27.749485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:27.750380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:27.750799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:27.751213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:27.752201 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:27.752652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:27.754852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:27.777600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94f186-93d2-4153-91aa-344c4e89a202 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:27.792317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 285/558] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:27.800037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:27.819707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:27.820363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:27.820632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:27.820902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:27.839263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:27.839263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:27.851027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:27.854172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:27.857055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:27.857055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:27.857055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:27.857055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 274/559] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:27.858389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:27.861431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:27.948647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:27.997826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321207e-c897-4ed5-95b4-031f3f00df2f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.002039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9a6a8-6ec0-42d4-9785-44006699d410 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.032742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e1019-fa9c-45f6-984c-97945dec8019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.033362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e1019-fa9c-45f6-984c-97945dec8019 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.033632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e1019-fa9c-45f6-984c-97945dec8019 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.033907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e1019-fa9c-45f6-984c-97945dec8019 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.034550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e1019-fa9c-45f6-984c-97945dec8019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:28.034852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e1019-fa9c-45f6-984c-97945dec8019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:28.036282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e1019-fa9c-45f6-984c-97945dec8019 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:28.040557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a5680-9fba-4312-b855-46e8a5c9dcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.041197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5680-9fba-4312-b855-46e8a5c9dcd7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.041455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5680-9fba-4312-b855-46e8a5c9dcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.041733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5680-9fba-4312-b855-46e8a5c9dcd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.049763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a5680-9fba-4312-b855-46e8a5c9dcd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:28.051257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a5680-9fba-4312-b855-46e8a5c9dcd7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:28.053693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 275/560] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:28.055075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19a62-00be-46e5-8f7e-95cd607cb9d5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.055399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19a62-00be-46e5-8f7e-95cd607cb9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.055662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19a62-00be-46e5-8f7e-95cd607cb9d5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.057184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 276/561] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:28.057893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f19a62-00be-46e5-8f7e-95cd607cb9d5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a46a5680-9fba-4312-b855-46e8a5c9dcd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:28.068376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:28.071627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 286/562] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd1d27-5bf4-4a79-b4ab-3ed3f43d783d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.079416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd1d27-5bf4-4a79-b4ab-3ed3f43d783d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.081667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd1d27-5bf4-4a79-b4ab-3ed3f43d783d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.096531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-34cd1d27-5bf4-4a79-b4ab-3ed3f43d783d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:28.098015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-34cd1d27-5bf4-4a79-b4ab-3ed3f43d783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:28.143568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:28.187817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 287/563] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:28.188907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd1d27-5bf4-4a79-b4ab-3ed3f43d783d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.203248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f19a62-00be-46e5-8f7e-95cd607cb9d5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.260914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.261957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.262475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.262929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.268698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:28.277536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:28.280520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:28.302383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:28.304499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 277/564] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:28.306633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.362371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd1d27-5bf4-4a79-b4ab-3ed3f43d783d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.398818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.399388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.399666 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.399934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.400687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:28.400998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:28.402430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:28.415027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 288/565] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:28.415896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.432797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37361ef7-dd0a-4a5d-9124-f45b19dd5b61 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.463150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:28.480506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7477f7-e987-4f37-b033-572111932f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.481652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7477f7-e987-4f37-b033-572111932f00 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.482088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7477f7-e987-4f37-b033-572111932f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.482789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7477f7-e987-4f37-b033-572111932f00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.500174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7477f7-e987-4f37-b033-572111932f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ce7477f7-e987-4f37-b033-572111932f00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:28.505452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:28.506576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:28.507435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 278/566] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:28.508126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7477f7-e987-4f37-b033-572111932f00 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.540453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760a932-d32e-4171-825c-05b6e0362aa7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.544118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:28.580232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.580833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.581109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.581378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.582116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:28.582422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:28.583905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:28.602278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 289/567] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:28.607458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.620289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7477f7-e987-4f37-b033-572111932f00 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.655094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.655553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.655832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.656106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.656827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=981e9da7f2774c90a667fab35ecab64b) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:28.657136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:28.658579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:28.663943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 279/568] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/role_assignments?user.id=981e9da7f2774c90a667fab35ecab64b&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:28.664712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.712975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe3e08-07dc-490d-b482-195c74db0c57 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.718634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:28.755167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78370992-09e4-4433-88b2-58fef066b99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.755880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370992-09e4-4433-88b2-58fef066b99d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.756141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370992-09e4-4433-88b2-58fef066b99d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.756412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370992-09e4-4433-88b2-58fef066b99d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.768986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78370992-09e4-4433-88b2-58fef066b99d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-78370992-09e4-4433-88b2-58fef066b99d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:28.774050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:28.775079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 290/569] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:28.780631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78370992-09e4-4433-88b2-58fef066b99d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.790433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870767b0-7f6c-41e8-b5bf-08872f2323ab None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.806884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:28.835498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162ad86-06a9-457e-aefe-39e238006708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.836284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ad86-06a9-457e-aefe-39e238006708 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.836570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ad86-06a9-457e-aefe-39e238006708 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.836845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ad86-06a9-457e-aefe-39e238006708 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.837509 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162ad86-06a9-457e-aefe-39e238006708 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:28.837811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162ad86-06a9-457e-aefe-39e238006708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:28.839304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162ad86-06a9-457e-aefe-39e238006708 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:28.852870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 280/570] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:28.852870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd39e7-9456-4b4a-a160-da92437fbc4d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.852870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd39e7-9456-4b4a-a160-da92437fbc4d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.852870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd39e7-9456-4b4a-a160-da92437fbc4d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.852870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 281/571] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:28.852870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd39e7-9456-4b4a-a160-da92437fbc4d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.900769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78370992-09e4-4433-88b2-58fef066b99d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.935931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:28.936489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:28.936762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:28.937030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:28.947344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:28.949321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:28.965499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:28.966707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:28.967960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:28.967960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:28.967960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:28.968541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 291/572] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:28.974248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:28.975111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd39e7-9456-4b4a-a160-da92437fbc4d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:28.985690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:29.018252 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf811e-74ad-42c7-867d-d21db84e91db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.018853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf811e-74ad-42c7-867d-d21db84e91db None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.019116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf811e-74ad-42c7-867d-d21db84e91db None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.019405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf811e-74ad-42c7-867d-d21db84e91db None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.020061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf811e-74ad-42c7-867d-d21db84e91db None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:29.020356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf811e-74ad-42c7-867d-d21db84e91db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.021764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf811e-74ad-42c7-867d-d21db84e91db None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.035015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 282/573] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:29.035015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21251daa-8b16-49ae-9873-c286699e7809 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.035015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21251daa-8b16-49ae-9873-c286699e7809 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.035015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21251daa-8b16-49ae-9873-c286699e7809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.052072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-21251daa-8b16-49ae-9873-c286699e7809 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:29.053288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-21251daa-8b16-49ae-9873-c286699e7809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:29.079928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:29.109579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a674242-8a1d-4ea2-9461-63179df49d40 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.142558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 283/574] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:29.147440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21251daa-8b16-49ae-9873-c286699e7809 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.155473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.156247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.156629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.156962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.169166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.170966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:29.189168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:29.190189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:29.191093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:29.191093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:29.191093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:29.193115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 292/575] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:29.194180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.289592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:29.294470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21251daa-8b16-49ae-9873-c286699e7809 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.320583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91484d-fb2a-446b-ae99-abd3c8e2719c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.331702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.332480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.332844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.333209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.334236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:29.334236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.335907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.348768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 284/576] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:29.349822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.355553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:29.367274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.368514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.368940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.369610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.383443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.385420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:29.404942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:29.406038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:29.406931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:29.406931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:29.406931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:29.411371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 293/577] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:29.412463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.474487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3162e43-eaf2-41cb-81be-783ba8ac3c63 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.511306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.511999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.512190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.512455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.513094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:29.513389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.514925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.517820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823ae23-563d-4119-a1cb-6bbf7e1e2e45 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.532394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 285/578] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:29.532394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.553890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:29.555884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.556516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.556893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.557238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.564498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.565937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:29.577845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:29.578751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 294/579] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:29.579703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.622328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:29.651976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ffef8-b5e3-47f4-bc6a-c086be3e46bf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.684546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8143fd-2b66-4103-8b81-68ca2d05b1eb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.692507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.693729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.694057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.694384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.695410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:29.695715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.697404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.709391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 286/580] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:29.714560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.721214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a194-5c22-4ab7-9cd7-318f3c88b78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.721848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a194-5c22-4ab7-9cd7-318f3c88b78b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.722123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a194-5c22-4ab7-9cd7-318f3c88b78b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.722397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a194-5c22-4ab7-9cd7-318f3c88b78b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.733511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0a194-5c22-4ab7-9cd7-318f3c88b78b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ecf0a194-5c22-4ab7-9cd7-318f3c88b78b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:29.738669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:29.739673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:29.740628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 295/581] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:29.741335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a194-5c22-4ab7-9cd7-318f3c88b78b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.810884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:29.832629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a64c7-d0a5-4b1d-88a0-65b95ec118b1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.845291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:29.850000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a194-5c22-4ab7-9cd7-318f3c88b78b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:29.868412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.869085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.869358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.869670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.882543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.884143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.886949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:29.887582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:29.887860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:29.888130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:29.891742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:29.891742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:29.891935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:29.897666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:29.898606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 287/582] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:29.904992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:29.916312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 296/583] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:29.916312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.032673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae74fb2-49c5-4b6f-bd20-1cdd2a8f7a3b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.036644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e09466-397e-4848-9b2b-ea2d0e8d9756 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.070474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.071437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.071757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.072001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.072629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:30.072926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.074286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.088181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 297/584] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:30.088372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.088929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.089096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.089208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.089475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.100014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.101898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.105671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:30.113996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:30.116742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:30.117707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 288/585] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:30.118650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.205583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374a211-d3c9-4c06-afc7-f3069c78ed22 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.228979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a97c7-2cc2-4aa4-85c5-103cee87a3b2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.242860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.243692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.244047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.244425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.245351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:30.245748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.247337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.266224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b52fc-fe71-4c5c-ab99-75727755dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.270808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 298/586] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 31 12:05:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:30.270989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b52fc-fe71-4c5c-ab99-75727755dff5 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.270989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b52fc-fe71-4c5c-ab99-75727755dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.270989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b52fc-fe71-4c5c-ab99-75727755dff5 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/0b113316bc3348b5a12c1d7b616bf4c9/roles/aafb8817e1d640eb961fa58db7e05111` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.271256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.287037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b52fc-fe71-4c5c-ab99-75727755dff5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=0b113316bc3348b5a12c1d7b616bf4c9, role_id=aafb8817e1d640eb961fa58db7e05111)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.288557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b52fc-fe71-4c5c-ab99-75727755dff5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.311273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 289/587] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:05:30 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/0b113316bc3348b5a12c1d7b616bf4c9/roles/aafb8817e1d640eb961fa58db7e05111 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:30.311273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b52fc-fe71-4c5c-ab99-75727755dff5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.368674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:30.383187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:30.401624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d30c0a-b27a-458d-86ed-065e76b863cd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.450679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.451611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.451890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.452157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.457119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b52fc-fe71-4c5c-ab99-75727755dff5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.464569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.466174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:30.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:30.486040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:30.487101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:30.487101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:30.487101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:30.487101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 299/588] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:30.487374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.503057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.504330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.504802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.505078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.506142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:30.506620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.508133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.540123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 290/589] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 31 12:05:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:30.542033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.603636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fd10d-1aef-420f-aef9-3d285cf5df6b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.635539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:30.642952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.643622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.643903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.644292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.656798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.658427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.664209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bd847-14ae-4b52-a27f-76e4791d87c4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:30.674248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:30.675499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:30.676520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:30.676520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:30.676520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:30.676693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:30.676766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 300/590] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:30.677269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.710834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b2059-a435-4fa6-8136-999af09de6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.711682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b2059-a435-4fa6-8136-999af09de6bc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.712141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b2059-a435-4fa6-8136-999af09de6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.712420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b2059-a435-4fa6-8136-999af09de6bc None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.713831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b2059-a435-4fa6-8136-999af09de6bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:30.713831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b2059-a435-4fa6-8136-999af09de6bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.714930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b2059-a435-4fa6-8136-999af09de6bc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.732015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 291/591] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:30.732015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3cb3-0677-4498-9a12-e7ad52fff905 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.732015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3cb3-0677-4498-9a12-e7ad52fff905 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.732015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3cb3-0677-4498-9a12-e7ad52fff905 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.732015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 292/592] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:30.732015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3cb3-0677-4498-9a12-e7ad52fff905 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.790321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95393b-5978-4c29-ac88-ac706ab72e3b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.830453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.831083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.831381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.831645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.842815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.844372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.846840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3cb3-0677-4498-9a12-e7ad52fff905 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:30.859526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:30.860552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:30.861433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:30.861433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:30.861433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:30.861433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 301/593] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:30.861433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:30.897646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5406ce-9d12-4034-8f74-b505e85b2abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:30.898276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5406ce-9d12-4034-8f74-b505e85b2abc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.898553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5406ce-9d12-4034-8f74-b505e85b2abc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.898827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5406ce-9d12-4034-8f74-b505e85b2abc None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.899510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5406ce-9d12-4034-8f74-b505e85b2abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:30.899813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5406ce-9d12-4034-8f74-b505e85b2abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:30.901231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5406ce-9d12-4034-8f74-b505e85b2abc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:30.905590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:30.919415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 293/594] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:30.919415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5e522-a57f-4353-936d-71f555f73886 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:30.919415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5e522-a57f-4353-936d-71f555f73886 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:30.919415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5e522-a57f-4353-936d-71f555f73886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:30.935446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3ae5e522-a57f-4353-936d-71f555f73886 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:30.937270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3ae5e522-a57f-4353-936d-71f555f73886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:30.962510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:30.983129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38046e5b-5323-4a6f-bb7f-d72d92bc4803 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.010043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 294/595] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:31.011013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5e522-a57f-4353-936d-71f555f73886 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.024738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.025716 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.026235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.026686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.035333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.037539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:31.050713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:31.051729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 12:05:31.052662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:31.055203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 302/596] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:31.056219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.145662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5e522-a57f-4353-936d-71f555f73886 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.156870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:31.164564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559014e8-86e5-4df2-bdc8-345bf6c187dd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.195419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.196372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.196721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.197451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.198433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:31.198816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.200340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.202835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.203758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.204203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.204639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.213115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.219996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 295/597] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:31.220192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.220289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:31.233191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:31.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:31.235479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:31.235479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:31.235479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:31.235479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:31.235479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:31.235479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:31.235479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:31.235479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:31.236959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 303/598] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:31.237889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.249557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:31.348302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12782ce-cc29-4222-976b-8866e7845696 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.348780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76271-034c-471e-99ad-8127a844d06f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.383505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.383842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.384120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.384395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.384512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.384668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.384758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.384966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.385384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:31.385688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.387141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.392632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.394075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.409305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 296/599] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:31.409305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:31.409549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:31.410350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 304/600] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:31.411217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.428297 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:31.499440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:31.521513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862294d6-e573-41a1-8917-c7fa409b4973 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.528888 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b2a98-3e9e-40a2-a864-375be1a41452 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.555412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc769-c080-4de6-9309-350fcdc2c46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.556086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc769-c080-4de6-9309-350fcdc2c46f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.556355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc769-c080-4de6-9309-350fcdc2c46f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.556605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc769-c080-4de6-9309-350fcdc2c46f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.568532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efc769-c080-4de6-9309-350fcdc2c46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:31.570732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.571375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.571655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.571926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.572615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:31.572919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.574350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-91efc769-c080-4de6-9309-350fcdc2c46f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:31.574450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:31.575556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:31.576728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 305/601] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:31.577700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc769-c080-4de6-9309-350fcdc2c46f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.592041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 297/602] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:31.592869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.658572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:31.687357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc769-c080-4de6-9309-350fcdc2c46f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.710946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6c1e2-5949-49cd-9ddd-f54424fe4fc9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.725766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.726493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.726835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.727169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.738263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.739840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.747663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.748401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.748688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.748958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.749657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:31.749965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.751446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.754991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:31.755901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:31.757028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:31.758140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:31.758140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:31.758140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:31.763933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 306/603] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:31.763933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.769924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 298/604] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:31.770768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.879185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1911-d140-4943-bf94-a1076b88eb84 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.880628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc7a7b-9675-4454-a944-ecccc0b5372d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:31.912449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:31.918191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621293e-cec6-466e-90bb-59851fcb254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.918868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621293e-cec6-466e-90bb-59851fcb254b None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.919144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621293e-cec6-466e-90bb-59851fcb254b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.919445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621293e-cec6-466e-90bb-59851fcb254b None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/db3393d3cb0e4bbcb21e403c431bf61b/roles/aafb8817e1d640eb961fa58db7e05111` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.923735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:31.924370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:31.924646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:31.924912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:31.925574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:31.925873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.927253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.934646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621293e-cec6-466e-90bb-59851fcb254b None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=db3393d3cb0e4bbcb21e403c431bf61b, role_id=aafb8817e1d640eb961fa58db7e05111)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:31.935957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 307/605] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:31.936616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:31.937565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621293e-cec6-466e-90bb-59851fcb254b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:31.952665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 299/606] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:05:31 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/db3393d3cb0e4bbcb21e403c431bf61b/roles/aafb8817e1d640eb961fa58db7e05111 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:31.953344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621293e-cec6-466e-90bb-59851fcb254b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:32.021583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:32.083442 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba8a5a-a68f-4f09-b466-638188e97566 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:32.092923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621293e-cec6-466e-90bb-59851fcb254b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:32.129267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:32.130139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.130571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.131016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.131846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:32.132297 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:32.133836 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:32.135815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69fb99-05fc-4271-807e-2098f491aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:32.136492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69fb99-05fc-4271-807e-2098f491aa85 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.136759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69fb99-05fc-4271-807e-2098f491aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.137523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69fb99-05fc-4271-807e-2098f491aa85 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/eef756349e5844e59d7b20072f63c41c/roles/aafb8817e1d640eb961fa58db7e05111` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.150226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 308/607] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:32.151657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:32.160553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69fb99-05fc-4271-807e-2098f491aa85 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=eef756349e5844e59d7b20072f63c41c, role_id=aafb8817e1d640eb961fa58db7e05111)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:32.162034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69fb99-05fc-4271-807e-2098f491aa85 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:32.182228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 300/608] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:05:31 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/eef756349e5844e59d7b20072f63c41c/roles/aafb8817e1d640eb961fa58db7e05111 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:32.182228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69fb99-05fc-4271-807e-2098f491aa85 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:32.259767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:32.334251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31382964-08b9-447f-99ca-f46cd95c2868 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:32.346122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:32.354305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69fb99-05fc-4271-807e-2098f491aa85 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:32.374170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01fdc2-73e3-4ceb-b5e7-bc3a9d8aef2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:32.375320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fdc2-73e3-4ceb-b5e7-bc3a9d8aef2b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.375801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fdc2-73e3-4ceb-b5e7-bc3a9d8aef2b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.376304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fdc2-73e3-4ceb-b5e7-bc3a9d8aef2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.389354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01fdc2-73e3-4ceb-b5e7-bc3a9d8aef2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:32.390801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01fdc2-73e3-4ceb-b5e7-bc3a9d8aef2b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:32.397777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc0381-1619-4834-80f2-2d8797a77df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:32.398923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc0381-1619-4834-80f2-2d8797a77df9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.399205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc0381-1619-4834-80f2-2d8797a77df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.399651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc0381-1619-4834-80f2-2d8797a77df9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3c01fdc2-73e3-4ceb-b5e7-bc3a9d8aef2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:32.406761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:32.407878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:32.408770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:32.408770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:32.408770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:32.408770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 309/609] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:32.409907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54c66c-539d-47be-b685-c4cfb2c70530 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.410223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54c66c-539d-47be-b685-c4cfb2c70530 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.410497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54c66c-539d-47be-b685-c4cfb2c70530 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.412030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 310/610] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:32.412694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc0381-1619-4834-80f2-2d8797a77df9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:32.414600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc0381-1619-4834-80f2-2d8797a77df9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:32.416423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54c66c-539d-47be-b685-c4cfb2c70530 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-bebc0381-1619-4834-80f2-2d8797a77df9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:32.428006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:32.428968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:32.430470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 301/611] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:32.430975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2533827-2f4b-4dbd-b810-a0bc2e227f7f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.430975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2533827-2f4b-4dbd-b810-a0bc2e227f7f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.431343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2533827-2f4b-4dbd-b810-a0bc2e227f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.450602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e2533827-2f4b-4dbd-b810-a0bc2e227f7f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:32.451877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e2533827-2f4b-4dbd-b810-a0bc2e227f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:32.529815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 302/612] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:32.530674 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2533827-2f4b-4dbd-b810-a0bc2e227f7f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:32.559699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:32.566781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54c66c-539d-47be-b685-c4cfb2c70530 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:32.605631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:32.607073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.607742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.608236 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.609081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:32.609579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:32.611250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:32.626235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 311/613] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:32.632246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:32.669731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:32.687264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2533827-2f4b-4dbd-b810-a0bc2e227f7f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:32.719551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:32.720177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.720506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.720929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.722452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:32.722763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:32.724576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:32.738252 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 303/614] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:32.745420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:32.842161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9976-1664-433a-96c3-ca70937ff55a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:32.909110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:32.910068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.910508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.910933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.920505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:32.922410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:32.922801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e77bb-9931-4484-9463-fede29aa00ba None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:32.936655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:32.937863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:32.939037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:32.939829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 312/615] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:32.940447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:32.966241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d7b6-3906-4ae4-83f2-9626084489f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:32.967554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d7b6-3906-4ae4-83f2-9626084489f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:32.968000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d7b6-3906-4ae4-83f2-9626084489f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:32.968481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d7b6-3906-4ae4-83f2-9626084489f4 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/ba8c7348ad2e457e80e1fdd2d93aacc6/roles/aafb8817e1d640eb961fa58db7e05111` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:32.971482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:32.987218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d7b6-3906-4ae4-83f2-9626084489f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=ba8c7348ad2e457e80e1fdd2d93aacc6, role_id=aafb8817e1d640eb961fa58db7e05111)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:32.989158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d7b6-3906-4ae4-83f2-9626084489f4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:32.992374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:33.006260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 304/616] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:05:32 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/ba8c7348ad2e457e80e1fdd2d93aacc6/roles/aafb8817e1d640eb961fa58db7e05111 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:05:33.008015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d7b6-3906-4ae4-83f2-9626084489f4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:33.061821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc612e8-53ca-4b61-9192-1989f8ee4377 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:33.099648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d53a2a-c543-4572-84d2-868d98dad488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:33.100341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d53a2a-c543-4572-84d2-868d98dad488 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:33.100621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d53a2a-c543-4572-84d2-868d98dad488 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:33.100894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d53a2a-c543-4572-84d2-868d98dad488 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:33.113189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d53a2a-c543-4572-84d2-868d98dad488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d4d53a2a-c543-4572-84d2-868d98dad488 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:33.118676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:33.119897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:33.121058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 313/617] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:33.125989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d53a2a-c543-4572-84d2-868d98dad488 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:33.137978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d7b6-3906-4ae4-83f2-9626084489f4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:33.183414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:33.184470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:33.184945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:33.185411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:33.186380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:33.186879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:33.188748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:33.209561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 305/618] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:33.217247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:33.220644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:33.263085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d53a2a-c543-4572-84d2-868d98dad488 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:33.301801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:33.302423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:33.302704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:33.302974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:33.303677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:33.303986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:33.305374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:33.316669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 314/619] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:33.322390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:33.323418 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:33.384858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c76e07-0c23-45c6-aa92-04da6bf05c3c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:33.435460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e8986-cb26-4d99-8d3c-1860731502ad None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:33.461411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:33.462058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:33.462336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:33.462578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:33.478574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:33.479037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:33.479709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:33.479987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:33.480257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:33.481085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:33.497005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:33.498388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:33.499556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:33.499556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:33.499556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:33.501392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 306/620] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:33.502262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:33.508781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:33.510516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:33.510875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:33.526951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:33.527969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:33.528873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:33.528873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:33.528873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:33.528873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 315/621] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:33.540344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:33.662521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee127b6f-cf9a-4130-bbed-2b84a4046ff3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:33.689500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882598d-b818-4dfd-b0a3-2c5dd18a372e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:33.698708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:33.699354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:33.699636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:33.699910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:33.700560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:33.700937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:33.702502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:33.702972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:33.715779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 316/622] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:33.722030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:33.765171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:33.765961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:33.766337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:33.766708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:33.768153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:33.769879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:33.771410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:33.785338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 307/623] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:33.790817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:33.816050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:33.861576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb5d67-86ca-48a9-a98b-2c01864649a0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:33.902756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:33.903464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:33.903743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:33.904051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:33.912336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:33.913794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:33.926751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:33.928011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:33.929380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 317/624] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:33.933589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:34.025389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f1045-d4a4-4416-a989-69d275d78e12 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:34.098809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:34.099413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.099691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.099964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.120244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:34.121733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:34.136097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49af4ac-2d93-40a1-b871-fb9c778e1779 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:34.150120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:34.151248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:34.152370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:34.152370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:34.152370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:34.152370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:34.152370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:34.152370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:34.152370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:34.152370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:34.153831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 308/625] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:34.158546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:34.167374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:34.205995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1a46e-405b-4857-9cb3-183835818094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:34.206606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a46e-405b-4857-9cb3-183835818094 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.206881 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a46e-405b-4857-9cb3-183835818094 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.207149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a46e-405b-4857-9cb3-183835818094 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.209247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1a46e-405b-4857-9cb3-183835818094 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:34.209555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1a46e-405b-4857-9cb3-183835818094 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:34.210991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1a46e-405b-4857-9cb3-183835818094 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:34.214806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:34.239265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 318/626] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:05:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:34.239265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568385fc-172d-49df-8562-222183dbde63 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.239265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568385fc-172d-49df-8562-222183dbde63 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.239265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568385fc-172d-49df-8562-222183dbde63 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.239265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 319/627] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:34.239265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568385fc-172d-49df-8562-222183dbde63 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:34.302993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2314a-5773-43c9-8d78-91b816622f7d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:34.338948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd46498-4d73-43c8-84b4-ce57e147f6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:34.339923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd46498-4d73-43c8-84b4-ce57e147f6cb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.340365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd46498-4d73-43c8-84b4-ce57e147f6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.340772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd46498-4d73-43c8-84b4-ce57e147f6cb None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.341855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd46498-4d73-43c8-84b4-ce57e147f6cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:34.342303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd46498-4d73-43c8-84b4-ce57e147f6cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:34.343957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd46498-4d73-43c8-84b4-ce57e147f6cb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:34.353050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568385fc-172d-49df-8562-222183dbde63 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:34.357962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 309/628] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:34.359474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f639-9e85-47fc-b190-b760786d3484 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.359917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f639-9e85-47fc-b190-b760786d3484 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.360323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f639-9e85-47fc-b190-b760786d3484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.379565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3b50f639-9e85-47fc-b190-b760786d3484 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:34.380997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3b50f639-9e85-47fc-b190-b760786d3484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:34.391570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7942e-cf50-4f32-a3e2-a05e70de2646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:34.392482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7942e-cf50-4f32-a3e2-a05e70de2646 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.392602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7942e-cf50-4f32-a3e2-a05e70de2646 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.392876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7942e-cf50-4f32-a3e2-a05e70de2646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.445417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:34.467710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 310/629] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:34.468763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50f639-9e85-47fc-b190-b760786d3484 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:34.473292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7942e-cf50-4f32-a3e2-a05e70de2646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:34.474824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7942e-cf50-4f32-a3e2-a05e70de2646 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:34.492912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:34.583994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50f639-9e85-47fc-b190-b760786d3484 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:34.594673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 320/630] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:34.595992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865354ac-3c4b-4495-ae4e-d6f5ef3332a5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.596276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865354ac-3c4b-4495-ae4e-d6f5ef3332a5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.596546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865354ac-3c4b-4495-ae4e-d6f5ef3332a5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.598379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 321/631] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:34.603029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865354ac-3c4b-4495-ae4e-d6f5ef3332a5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:34.628265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57cad1-2c3e-4106-ad68-1fe4445458a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:34.629177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57cad1-2c3e-4106-ad68-1fe4445458a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.629556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57cad1-2c3e-4106-ad68-1fe4445458a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.629928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57cad1-2c3e-4106-ad68-1fe4445458a9 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/981e9da7f2774c90a667fab35ecab64b/roles/aafb8817e1d640eb961fa58db7e05111` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.646733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57cad1-2c3e-4106-ad68-1fe4445458a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=981e9da7f2774c90a667fab35ecab64b, role_id=aafb8817e1d640eb961fa58db7e05111)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:34.648503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57cad1-2c3e-4106-ad68-1fe4445458a9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:34.666513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 311/632] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:05:34 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/981e9da7f2774c90a667fab35ecab64b/roles/aafb8817e1d640eb961fa58db7e05111 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:34.668386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3d5c-9616-4517-8346-21cb3ddbf3fe None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.668964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3d5c-9616-4517-8346-21cb3ddbf3fe None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.669298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3d5c-9616-4517-8346-21cb3ddbf3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.683784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:34.691209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-613c3d5c-9616-4517-8346-21cb3ddbf3fe None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:34.692543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-613c3d5c-9616-4517-8346-21cb3ddbf3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:34.723471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865354ac-3c4b-4495-ae4e-d6f5ef3332a5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:34.764106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:34.779791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 312/633] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:34.781269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c3d5c-9616-4517-8346-21cb3ddbf3fe None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:34.784076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:34.791267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:34.791267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:34.791267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:34.798945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:34.800833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:34.819357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:34.820558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:34.821651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:34.821651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:34.821651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:34.827272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 322/634] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:34.830063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:34.975875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c3d5c-9616-4517-8346-21cb3ddbf3fe None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.032558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1278-c333-4c91-8ddc-f1a7db9f3958 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.043200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b008e0-777e-4cb2-9e4f-1b20ef0e3a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.043881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b008e0-777e-4cb2-9e4f-1b20ef0e3a78 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.044159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b008e0-777e-4cb2-9e4f-1b20ef0e3a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.044434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b008e0-777e-4cb2-9e4f-1b20ef0e3a78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.069175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b008e0-777e-4cb2-9e4f-1b20ef0e3a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-72b008e0-777e-4cb2-9e4f-1b20ef0e3a78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:35.074226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:35.075365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:35.076454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:35.079550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:35.083275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 313/635] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:35.087145 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b008e0-777e-4cb2-9e4f-1b20ef0e3a78 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.096457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.097077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.100191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.100513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.101200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:35.101512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.102942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:35.125545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 323/636] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:35.126425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.175836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:35.223406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b008e0-777e-4cb2-9e4f-1b20ef0e3a78 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.237093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebb908-4b34-4c26-9ead-c4a766b4b60f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.266166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.266797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.267077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.267384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.268329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:35.268639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.270853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:35.278801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0710844-5285-46ed-b787-da21a39459fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.280514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0710844-5285-46ed-b787-da21a39459fa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.280869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 314/637] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:35.281226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0710844-5285-46ed-b787-da21a39459fa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.281705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.281917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0710844-5285-46ed-b787-da21a39459fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.294219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0710844-5285-46ed-b787-da21a39459fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e0710844-5285-46ed-b787-da21a39459fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:35.300647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:35.301634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:35.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:35.309729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 324/638] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:35.309729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0710844-5285-46ed-b787-da21a39459fa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.367735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:35.398309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb1811-6f05-469f-a3bb-3ced32573426 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.413670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:35.431563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.432122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.432401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.432671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.434085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0710844-5285-46ed-b787-da21a39459fa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.439716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.441178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:35.453493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:35.454589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 315/639] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:35.455530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.481473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b4428-d3f8-4b14-b6b4-e5ca1fc35811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.483274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b4428-d3f8-4b14-b6b4-e5ca1fc35811 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.483816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b4428-d3f8-4b14-b6b4-e5ca1fc35811 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.484231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b4428-d3f8-4b14-b6b4-e5ca1fc35811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.495561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b4428-d3f8-4b14-b6b4-e5ca1fc35811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.497198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b4428-d3f8-4b14-b6b4-e5ca1fc35811 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-878b4428-d3f8-4b14-b6b4-e5ca1fc35811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:35.514258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:35.515504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:35.516551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:35.516551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:35.516551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:35.518398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 325/640] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:35.519847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7906d2-cf29-4504-9c14-6e733a595b1a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.520128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7906d2-cf29-4504-9c14-6e733a595b1a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.520401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7906d2-cf29-4504-9c14-6e733a595b1a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.521831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 326/641] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:35.522390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7906d2-cf29-4504-9c14-6e733a595b1a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.569776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e70977-1883-4f47-940d-8bd87c9718f8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.606945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.607632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.607903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.608162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.608822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:35.609122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.610490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:35.623542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 316/642] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:35.623542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.639938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:35.647556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7906d2-cf29-4504-9c14-6e733a595b1a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.669908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:35.683892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a619a40-d1c7-4974-83ca-dfafb4c2f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.684418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a619a40-d1c7-4974-83ca-dfafb4c2f327 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.684691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a619a40-d1c7-4974-83ca-dfafb4c2f327 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.684962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a619a40-d1c7-4974-83ca-dfafb4c2f327 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.685723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a619a40-d1c7-4974-83ca-dfafb4c2f327 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:35.686023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a619a40-d1c7-4974-83ca-dfafb4c2f327 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.687476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a619a40-d1c7-4974-83ca-dfafb4c2f327 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:35.704851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 327/643] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:35.704851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fce99b-ca22-4adf-b965-7a6684deb6ed None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.704851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fce99b-ca22-4adf-b965-7a6684deb6ed None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.704851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fce99b-ca22-4adf-b965-7a6684deb6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.720265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-88fce99b-ca22-4adf-b965-7a6684deb6ed None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:35.721519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-88fce99b-ca22-4adf-b965-7a6684deb6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:35.739547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f62e0-d408-44e6-85dd-620b59522a6f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.776872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.777771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.778051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.778332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.779096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:35.779550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.780947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:35.794286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 317/644] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:35.795937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.811316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 328/645] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:35.811316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fce99b-ca22-4adf-b965-7a6684deb6ed None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:35.920626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:35.924225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00744-3c33-4a2f-8187-df7191319b0b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.929613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:35.950322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fce99b-ca22-4adf-b965-7a6684deb6ed None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:35.965116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.965886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.966262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.966537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.980877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.982447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:35.996115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:35.996727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:35.996999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:35.997266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:35.998007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=0b113316bc3348b5a12c1d7b616bf4c9) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:35.998310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:35.999824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:36.005052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:36.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:36.007059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:36.007059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:36.007059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:36.014342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 329/646] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/role_assignments?user.id=0b113316bc3348b5a12c1d7b616bf4c9&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:36.014342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.015141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 318/647] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:36.016335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.148119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03608d7e-fd3a-4f91-ae5f-990455b63f8d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.185805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271720b-fadb-4194-aeaa-dc500ebd461c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.188892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.189510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.189784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.190051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.190686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:36.190982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:36.192413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.205874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 319/648] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:36.206886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.223758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.224407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.224665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.224933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.225893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:36.236050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:36.237706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:36.252928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:36.254108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 12:05:36.255253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:36.255949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 330/649] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:36.260151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.279514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:36.344105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e747fdf-e1cc-48c7-98e2-50fd292f8dc0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.382907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7dd91-4745-46e6-8ebb-0f4ffaa1d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.383832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7dd91-4745-46e6-8ebb-0f4ffaa1d350 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.384256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7dd91-4745-46e6-8ebb-0f4ffaa1d350 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.384745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7dd91-4745-46e6-8ebb-0f4ffaa1d350 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.388421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e801966-00bb-49bb-96db-a9a827031ae6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.396812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7dd91-4745-46e6-8ebb-0f4ffaa1d350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-46b7dd91-4745-46e6-8ebb-0f4ffaa1d350 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:36.402330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:36.403504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:36.406715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:36.409831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 320/650] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:36.410945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7dd91-4745-46e6-8ebb-0f4ffaa1d350 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.434384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.435370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.435949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.435949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.444514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:36.445966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:36.458346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:36.459338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:36.460284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 331/651] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:36.460884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.502020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:36.536902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7dd91-4745-46e6-8ebb-0f4ffaa1d350 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.551648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:36.575310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.576005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.577050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.577325 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.578133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:36.578429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:36.580061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.592869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb28790-0339-4b83-9634-fd5fb9097670 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.606147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 321/652] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:36.607217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.634838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.635955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.636439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.636912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.637774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:36.638429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:36.640094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.653641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 332/653] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:36.654967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.745389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7e1a1-46cf-43b7-abd7-e7fd710943b6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.767491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af799a7-6a0d-41cf-8182-1737d5ebc094 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.777111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:36.784596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.785215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.785497 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.785765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.786484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:36.786784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:36.788258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.799961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:36.804993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59b79b-5b57-4693-9b66-5c831b5f2e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.806121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59b79b-5b57-4693-9b66-5c831b5f2e99 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.806503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59b79b-5b57-4693-9b66-5c831b5f2e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.806781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59b79b-5b57-4693-9b66-5c831b5f2e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.807870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 322/654] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:36.808565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:36.888041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59b79b-5b57-4693-9b66-5c831b5f2e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:36.890268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59b79b-5b57-4693-9b66-5c831b5f2e99 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.932415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06919ea8-eae0-4b96-947b-2db8d2b550fa None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:36.966395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b49ec4-bd24-438e-8793-276dd805e352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:36.967202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49ec4-bd24-438e-8793-276dd805e352 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:36.967516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49ec4-bd24-438e-8793-276dd805e352 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:36.967785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49ec4-bd24-438e-8793-276dd805e352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:36.979506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b49ec4-bd24-438e-8793-276dd805e352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:36.981241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b49ec4-bd24-438e-8793-276dd805e352 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b7b49ec4-bd24-438e-8793-276dd805e352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:36.997506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:36.998541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:36.999928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:36.999928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:36.999928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:36.999928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 323/655] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:37.000532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99060-7da6-45d0-9619-29af9a00a650 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.000816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99060-7da6-45d0-9619-29af9a00a650 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.001114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99060-7da6-45d0-9619-29af9a00a650 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.002646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 324/656] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:37.004069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc99060-7da6-45d0-9619-29af9a00a650 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.010071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 333/657] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:37.012285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59b79b-5b57-4693-9b66-5c831b5f2e99 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.049206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:37.056486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:37.124598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59b79b-5b57-4693-9b66-5c831b5f2e99 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:37.131343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc99060-7da6-45d0-9619-29af9a00a650 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:37.161955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605d8a5-5863-4d75-930a-3c5bd3834429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:37.162731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605d8a5-5863-4d75-930a-3c5bd3834429 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.163084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605d8a5-5863-4d75-930a-3c5bd3834429 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.163482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605d8a5-5863-4d75-930a-3c5bd3834429 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.164278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605d8a5-5863-4d75-930a-3c5bd3834429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:37.164671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605d8a5-5863-4d75-930a-3c5bd3834429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:37.166130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605d8a5-5863-4d75-930a-3c5bd3834429 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:37.167310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5b2a-7a72-4ccf-acc6-bd96dafe24fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:37.167934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5b2a-7a72-4ccf-acc6-bd96dafe24fb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.168209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5b2a-7a72-4ccf-acc6-bd96dafe24fb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.168480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5b2a-7a72-4ccf-acc6-bd96dafe24fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.183719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 334/658] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:37.185212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36795f-d958-4b6d-8dde-a15449f2bcc4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.185934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36795f-d958-4b6d-8dde-a15449f2bcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.186337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36795f-d958-4b6d-8dde-a15449f2bcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.207949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0e36795f-d958-4b6d-8dde-a15449f2bcc4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:37.209141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0e36795f-d958-4b6d-8dde-a15449f2bcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:37.248784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db5b2a-7a72-4ccf-acc6-bd96dafe24fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:37.250263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db5b2a-7a72-4ccf-acc6-bd96dafe24fb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:37.282023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 335/659] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:37.283597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36795f-d958-4b6d-8dde-a15449f2bcc4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.300041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:37.325105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:37.366231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 325/660] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:37.367334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5b2a-7a72-4ccf-acc6-bd96dafe24fb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.428243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36795f-d958-4b6d-8dde-a15449f2bcc4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:37.462463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:37.463091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.463395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.463661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.474026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:37.475518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:37.478831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5b2a-7a72-4ccf-acc6-bd96dafe24fb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:37.490151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:37.491165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:37.492146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:37.492146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:37.492146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:37.492146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 336/661] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:37.492146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.521470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:37.522684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.523257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.523772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.532244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:37.534022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:37.539139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:37.549828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:37.550921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 12:05:37.551922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:37.558816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 326/662] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:37.560016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.578384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:37.608038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4636170-f252-42dc-a879-c1cb28a92e90 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:37.644078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:37.644705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.644978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.645400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.656311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:37.657762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:37.669270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228dbd3-f6e3-4ec0-9ef1-e6f3a4385c7a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:37.672508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:37.673565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:37.675047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:37.675047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:37.675047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:37.675047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 337/663] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:37.675047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.719080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:37.719723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.720002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.720274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.720909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:37.721214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:37.722607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:37.739878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 327/664] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:37.739878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.799691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:37.799968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e70a35-5003-47c3-8c36-301f50e68389 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:37.830012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:37.840932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:37.841827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.842387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.842762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.843304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0a148-397c-4cb7-a27c-d6445d13b1f3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:37.843667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:37.844041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:37.845551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:37.868744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 338/665] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:37.868744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.895561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a6c3b-86c5-4b88-8d8f-f4df0495433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:37.896275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a6c3b-86c5-4b88-8d8f-f4df0495433b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:37.896555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a6c3b-86c5-4b88-8d8f-f4df0495433b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:37.896827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a6c3b-86c5-4b88-8d8f-f4df0495433b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:37.908877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a6c3b-86c5-4b88-8d8f-f4df0495433b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ac5a6c3b-86c5-4b88-8d8f-f4df0495433b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:37.914216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:37.915184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:37.916134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:37.922673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 328/666] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:37.922673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a6c3b-86c5-4b88-8d8f-f4df0495433b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:37.988317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610322be-c1fd-4946-8143-96b0399692d9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.026119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.026734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.027014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.027315 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.027963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:38.028269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.029851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.034269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a6c3b-86c5-4b88-8d8f-f4df0495433b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.047493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 339/667] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:38.047493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.056594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:38.073123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.073760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.074040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.074307 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.081542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.083016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:38.094029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:38.100294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 329/668] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:38.101528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.102639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:38.151204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ffebd-de5a-4a02-acfa-0879af310950 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.188980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea916d-964a-4ba5-ade2-881c08362cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.189588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea916d-964a-4ba5-ade2-881c08362cda None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.189863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea916d-964a-4ba5-ade2-881c08362cda None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.190129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea916d-964a-4ba5-ade2-881c08362cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.199813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a657f1c-c033-47f1-b0e9-16d7f8c37dd9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.243409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.244361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.244677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.244869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.245530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:38.245868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.247339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.255521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 330/669] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:38.256318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.264087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea916d-964a-4ba5-ade2-881c08362cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.265565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea916d-964a-4ba5-ade2-881c08362cda None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.302183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:38.313415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:38.361059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3af20c-abab-4d35-8652-807a3129acee None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.370822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 340/670] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:38.372186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b4f8-cc6e-4aa6-b04d-33f296ff332a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.372477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b4f8-cc6e-4aa6-b04d-33f296ff332a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.372800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b4f8-cc6e-4aa6-b04d-33f296ff332a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.374036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 341/671] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:38.374724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8b4f8-cc6e-4aa6-b04d-33f296ff332a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.402146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.402773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.403048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.403363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.410078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.411561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:38.422951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:38.424129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 331/672] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:38.425318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.483566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8b4f8-cc6e-4aa6-b04d-33f296ff332a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.518466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.519075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.519383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.519651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.520351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:38.520652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.522416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.538983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042d30-4419-43c7-9170-8e3f02e2196d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.540623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:38.541124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 342/673] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:38.542550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.557745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:38.591831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f5878-b18a-4584-b51b-84c4f95d09c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.592404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f5878-b18a-4584-b51b-84c4f95d09c7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.592682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f5878-b18a-4584-b51b-84c4f95d09c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.592955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f5878-b18a-4584-b51b-84c4f95d09c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.613088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f5878-b18a-4584-b51b-84c4f95d09c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.616824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f5878-b18a-4584-b51b-84c4f95d09c7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8a0f5878-b18a-4584-b51b-84c4f95d09c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:38.634945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:38.636212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:38.637303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:38.637303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:38.637303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:38.641119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 332/674] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:38.642642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c97b-fd55-44fa-bc41-fda1882d394f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.642917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c97b-fd55-44fa-bc41-fda1882d394f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.643196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c97b-fd55-44fa-bc41-fda1882d394f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.662732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dc09c97b-fd55-44fa-bc41-fda1882d394f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:38.664296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dc09c97b-fd55-44fa-bc41-fda1882d394f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:38.687704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b002a-d81c-48ec-b018-2028ff62994b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.726487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.727222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.727617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.728430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.740055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.741599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.747368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 333/675] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:38.748808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09c97b-fd55-44fa-bc41-fda1882d394f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:38.776659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:38.777862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:38.779129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:38.779129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:38.779129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:38.780967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 343/676] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:38.782064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.850167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:38.867726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:38.892209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09c97b-fd55-44fa-bc41-fda1882d394f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.899064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dbde0-0c87-4646-9b1c-2dcfc38a939b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:38.934212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.935757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.935757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.935757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.937005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:38.937118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:38.937230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.937624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:38.937981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:38.938295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:38.938852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.939144 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=db3393d3cb0e4bbcb21e403c431bf61b) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:38.939540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:38.941036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:38.960633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 334/677] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:38.960633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:38.960926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 344/678] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/role_assignments?user.id=db3393d3cb0e4bbcb21e403c431bf61b&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:38.960926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.091336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddc2ba-9bdc-40e2-ad08-748f35c9b133 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.102838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb38c3d-a589-4374-be06-b8141512025e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.120008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:39.141634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.142240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.142518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.142788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.143490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:39.143791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:39.145172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:39.147129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.147129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.147129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.147129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.147524 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:39.147768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:39.149159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:39.161976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 345/679] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:39.162187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 335/680] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:39.162924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.163653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.166836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:39.286584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c704a9-3efd-4130-81f8-150c78cebf2a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.289446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00004276-ff8b-4d17-8591-01534dfc6a8d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.330971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.331612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.331887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.332154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.332258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2472f-dc5c-42fa-9738-60602382a5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.332813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2472f-dc5c-42fa-9738-60602382a5cd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.332922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=eef756349e5844e59d7b20072f63c41c) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:39.333098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2472f-dc5c-42fa-9738-60602382a5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.333198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:39.333346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2472f-dc5c-42fa-9738-60602382a5cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.334604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:39.355577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 336/681] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/role_assignments?user.id=eef756349e5844e59d7b20072f63c41c&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:39.355577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2472f-dc5c-42fa-9738-60602382a5cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9cf2472f-dc5c-42fa-9738-60602382a5cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:39.355837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:39.356635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:39.357984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 346/682] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:39.358852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2472f-dc5c-42fa-9738-60602382a5cd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.408611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:39.464465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:39.488875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2472f-dc5c-42fa-9738-60602382a5cd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.520445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd8422-7817-488f-aa3f-7290110e5cd9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.533049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.533790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.534071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.534338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.542448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:39.543993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:39.557278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:39.558298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 347/683] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:39.559347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.565885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.566538 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.566877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.567667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.575498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:39.577028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:39.589902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:39.590868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 337/684] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:39.591936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.692673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:39.695832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044cf03-3446-4021-8584-d4310d1ceaa8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.728513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef96d-9ecb-4066-81df-bf90167ce6f6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.739833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.741175 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.741630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.742352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.743599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:39.744075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:39.745709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:39.761521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 348/685] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:39.762864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.768727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:39.789259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.789962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.790299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.790791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.792821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:39.793218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:39.795810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:39.818209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 338/686] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:39.819243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.912652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0454c-577d-4441-8c24-5fbce679e9bc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.952243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070574-5c36-4332-9d6a-d08c6dd4d263 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:39.954464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.955260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.955672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.956034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:39.956859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:39.957312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:39.958847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:39.973502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 349/687] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:39.974542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:39.998224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:39.999119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:39.999426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:39.999698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:40.003342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:40.017654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:40.020269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:40.043641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:40.044894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:40.045965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:40.045965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:40.045965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:40.048955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 339/688] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:40.050684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:40.098292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d138a8-3531-4093-9a20-4f19c5de9291 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:40.105356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:40.142494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ab3d2-bf6d-4e9b-9b6c-bd6a11ec687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:40.143350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ab3d2-bf6d-4e9b-9b6c-bd6a11ec687f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:40.143741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ab3d2-bf6d-4e9b-9b6c-bd6a11ec687f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:40.144122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ab3d2-bf6d-4e9b-9b6c-bd6a11ec687f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:40.187867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffcb77-ae01-42ef-a71f-5bea686c7952 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:40.216249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ab3d2-bf6d-4e9b-9b6c-bd6a11ec687f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:40.217893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ab3d2-bf6d-4e9b-9b6c-bd6a11ec687f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:40.227109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:40.227764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:40.228041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:40.228306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:40.229029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=ba8c7348ad2e457e80e1fdd2d93aacc6) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:40.229442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:40.231064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:40.238292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 340/689] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:40 2022] GET /identity/v3/role_assignments?user.id=ba8c7348ad2e457e80e1fdd2d93aacc6&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:40.239191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:40.244427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:40.334470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 350/690] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:40.340750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ab3d2-bf6d-4e9b-9b6c-bd6a11ec687f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:40.407985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dee11-bbd6-42ed-8468-a6b3524a9da8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:40.412008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:40.482979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:40.483356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:40.484375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:40.484375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:40.484750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:40.485127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:40.486697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:40.500175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 341/691] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:40.509940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:40.523079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ab3d2-bf6d-4e9b-9b6c-bd6a11ec687f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:40.550106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:40.577182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:40.577809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:40.578093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:40.578365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:40.594395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:40.596235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:40.619335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:40.620464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:40.621364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:40.621364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:40.621364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:40.621364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 351/692] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:40.626666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67522a7-fbd0-4608-8980-74805508b8d6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:40.628037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:40.676437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:40.677511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:40.677790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:40.678132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:40.687603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:40.689642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:40.703113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:40.704295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:40.705245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:40.707869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 342/693] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:05:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:40.713850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:40.743286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:40.767042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7a2eb-66d8-4a73-8756-78e765893abc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:40.808017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:40.809021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:40.809489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:40.810098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:40.811258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:40.811794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:40.813725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:40.828533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 352/694] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:40.836426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:40.914072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd7d78-a836-4db1-9908-a43e584e1632 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:40.918777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:40.953230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:40.954302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:40.954868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:40.955426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:40.956394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:40.956943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:40.958683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:40.980397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 343/695] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:05:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:40.986135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:41.058000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952527-e01a-46db-abab-7ff6300f870e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:41.128726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90a58e-060c-4f25-b3bd-087ba24ef40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:41.129374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90a58e-060c-4f25-b3bd-087ba24ef40c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.129649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90a58e-060c-4f25-b3bd-087ba24ef40c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.129920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90a58e-060c-4f25-b3bd-087ba24ef40c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.130218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:41.138510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90a58e-060c-4f25-b3bd-087ba24ef40c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:41.140208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90a58e-060c-4f25-b3bd-087ba24ef40c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:41.152381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02410f12-20e4-4c8f-af2f-ecdc3a7150f8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ea90a58e-060c-4f25-b3bd-087ba24ef40c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:41.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:41.154056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:41.154942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:41.154942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:41.154942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:41.154942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:41.154942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:41.154942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:41.154942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:41.154942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:41.158856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 353/696] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:41.160320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca232e9-d502-4acb-8387-a9d1cb8d3cea None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.160703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca232e9-d502-4acb-8387-a9d1cb8d3cea None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.160987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca232e9-d502-4acb-8387-a9d1cb8d3cea None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.163674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 354/697] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:41.164779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca232e9-d502-4acb-8387-a9d1cb8d3cea None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:41.194326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010713f-bfb5-462c-9725-4d69b422686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:41.194946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010713f-bfb5-462c-9725-4d69b422686b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.195222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010713f-bfb5-462c-9725-4d69b422686b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.195520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010713f-bfb5-462c-9725-4d69b422686b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.272452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010713f-bfb5-462c-9725-4d69b422686b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:41.273592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:41.274726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010713f-bfb5-462c-9725-4d69b422686b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:41.284554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca232e9-d502-4acb-8387-a9d1cb8d3cea None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:41.323048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1dc93-11d6-42b3-a6a2-599f19e200f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:41.323616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1dc93-11d6-42b3-a6a2-599f19e200f7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.323894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1dc93-11d6-42b3-a6a2-599f19e200f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.324164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1dc93-11d6-42b3-a6a2-599f19e200f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.324967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1dc93-11d6-42b3-a6a2-599f19e200f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:41.325277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1dc93-11d6-42b3-a6a2-599f19e200f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:41.327504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1dc93-11d6-42b3-a6a2-599f19e200f7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:41.346655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 355/698] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:41.348407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab1669-ae09-454f-b935-c36a3ac5a2bf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.348691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab1669-ae09-454f-b935-c36a3ac5a2bf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.348966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab1669-ae09-454f-b935-c36a3ac5a2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.362566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:41.371663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-44ab1669-ae09-454f-b935-c36a3ac5a2bf None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:41.373241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-44ab1669-ae09-454f-b935-c36a3ac5a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:41.389550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 344/699] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:41.390472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010713f-bfb5-462c-9725-4d69b422686b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:41.452841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 356/700] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:41.454095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab1669-ae09-454f-b935-c36a3ac5a2bf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:41.546748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010713f-bfb5-462c-9725-4d69b422686b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:41.571820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:41.579784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fdc10-9d9b-4529-9b5a-a0a7a853fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:41.580742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fdc10-9d9b-4529-9b5a-a0a7a853fff5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.581279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fdc10-9d9b-4529-9b5a-a0a7a853fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.581692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fdc10-9d9b-4529-9b5a-a0a7a853fff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.582569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fdc10-9d9b-4529-9b5a-a0a7a853fff5 None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=981e9da7f2774c90a667fab35ecab64b) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:41.583021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fdc10-9d9b-4529-9b5a-a0a7a853fff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:41.584659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fdc10-9d9b-4529-9b5a-a0a7a853fff5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:41.591728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 345/701] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:41 2022] GET /identity/v3/role_assignments?user.id=981e9da7f2774c90a667fab35ecab64b&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:41.611144 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab1669-ae09-454f-b935-c36a3ac5a2bf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:41.669089 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:41.670614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.671091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.671538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.691792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:41.693508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:41.717702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:41.718980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:41.720230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:41.720230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:41.720230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:41.721827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 357/702] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:41.728553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c221a-fbfd-4f86-92da-c547eada710c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.728897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c221a-fbfd-4f86-92da-c547eada710c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.729091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c221a-fbfd-4f86-92da-c547eada710c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.730638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 346/703] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:41.731646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:41.737183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5273925-027d-464c-adbc-a9f71f222765 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.737819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5273925-027d-464c-adbc-a9f71f222765 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.738298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5273925-027d-464c-adbc-a9f71f222765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.765017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e5273925-027d-464c-adbc-a9f71f222765 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:41.766219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e5273925-027d-464c-adbc-a9f71f222765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:41.822559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:41.862416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 347/704] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 12:05:41.862416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5273925-027d-464c-adbc-a9f71f222765 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:41.877309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2635-b2b1-40c5-a046-54e6d5a5a03f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:41.939479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:41.940131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:41.940418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:41.940691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:41.942040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:41.942040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:41.944023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:41.948547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:41.966127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 358/705] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:41.972266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:42.001350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5273925-027d-464c-adbc-a9f71f222765 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:42.047541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:42.048422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.048700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.048972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.049566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:42.053005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:42.066391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:42.067581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:42.068478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 348/706] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 31 12:05:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:42.073775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:42.128187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19d0a-9c77-4051-85d9-27e457bd0a57 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:42.139614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:42.170239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:42.171443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.172041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.172548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.182373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:42.185695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:42.200709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:42.201662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:42.202598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:42.202598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:42.202598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:42.202598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:42.202598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:42.202598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:42.202598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:42.202598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:42.205370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 359/707] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:42.211309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:42.221192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01298da6-8b5c-497a-8899-0a1f0972fefa None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:42.270019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:42.270654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.270937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.271211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.271877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:42.272241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:42.275015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:42.281245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 349/708] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 31 12:05:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:42.282613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:42.299575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:42.352104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312625d-5a81-4a42-aaea-d8720aa39512 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:42.391893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:42.392817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df6b87-a1cc-4152-aaac-4d9ceb3f7f90 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:42.393073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.393533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.394030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.394446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:42.396288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:42.396857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:42.398615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:42.414462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 360/709] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:42.415920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:42.431886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:42.432547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.432830 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.433101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.447578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:42.449267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:42.469664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:42.470714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:42.471740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:42.471740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:42.471740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:42.471740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 350/710] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:42.472011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:42.557989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b906294-9a9f-4ffc-8b6d-b332b50c6026 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:42.584455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:42.593222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf3d65-8b87-4cd0-ba53-a2cd6e085274 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:42.609378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:42.609985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.610260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.610534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.611166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:42.611508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:42.612897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:42.627267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 361/711] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 31 12:05:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:42.627267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:42.644908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9ab56-221b-4e05-b973-87fe1ef36f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:42.645468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9ab56-221b-4e05-b973-87fe1ef36f16 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.645748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9ab56-221b-4e05-b973-87fe1ef36f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.646019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9ab56-221b-4e05-b973-87fe1ef36f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.646773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9ab56-221b-4e05-b973-87fe1ef36f16 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=0b113316bc3348b5a12c1d7b616bf4c9) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:42.647084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9ab56-221b-4e05-b973-87fe1ef36f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:42.648699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9ab56-221b-4e05-b973-87fe1ef36f16 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:42.661873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 351/712] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:42 2022] GET /identity/v3/role_assignments?user.id=0b113316bc3348b5a12c1d7b616bf4c9&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:42.663819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05da81e-eead-423e-8db9-44f8b68a88f9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.664118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05da81e-eead-423e-8db9-44f8b68a88f9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.664393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05da81e-eead-423e-8db9-44f8b68a88f9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.665844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 352/713] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:42.675548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53156744-a1d1-4c6c-8f67-d7a53c35782a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.676097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53156744-a1d1-4c6c-8f67-d7a53c35782a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.676602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53156744-a1d1-4c6c-8f67-d7a53c35782a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.733535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-53156744-a1d1-4c6c-8f67-d7a53c35782a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:42.741431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-53156744-a1d1-4c6c-8f67-d7a53c35782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:42.792955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:42.886338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45597e90-e191-450f-9d58-b2d0ea779ddd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:42.896287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 353/714] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:42.949939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:42.951009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:42.951581 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:42.952146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:42.953038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:42.953542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:42.955137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:42.978772 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 362/715] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:42.984324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53156744-a1d1-4c6c-8f67-d7a53c35782a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:43.104955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53156744-a1d1-4c6c-8f67-d7a53c35782a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:43.140946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71bcf8-c89f-4117-b9ab-e1040a49b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:43.141555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71bcf8-c89f-4117-b9ab-e1040a49b066 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:43.141829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71bcf8-c89f-4117-b9ab-e1040a49b066 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:43.142095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71bcf8-c89f-4117-b9ab-e1040a49b066 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:43.142817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71bcf8-c89f-4117-b9ab-e1040a49b066 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:43.144431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71bcf8-c89f-4117-b9ab-e1040a49b066 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:43.177640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 354/716] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:05:42 2022] POST /identity/v3/users => generated 254 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:43.316353 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:43.363872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:43.444290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0375c-bee1-473c-b3e8-ef7e9e6c5673 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:43.484248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:43.484897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:43.485172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:43.485440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:43.486080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:43.487470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:43.502026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:43.503092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:43.504582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 363/717] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:43 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:43.509279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71bcf8-c89f-4117-b9ab-e1040a49b066 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:43.514361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:43.626387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71bcf8-c89f-4117-b9ab-e1040a49b066 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:43.666060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af5606-09ee-4909-977b-2d730eb89c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:43.666685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5606-09ee-4909-977b-2d730eb89c95 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:43.666958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5606-09ee-4909-977b-2d730eb89c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:43.667222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5606-09ee-4909-977b-2d730eb89c95 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:43.667886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af5606-09ee-4909-977b-2d730eb89c95 None admin] RBAC: Adding query filter params (name=network) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:43.668181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af5606-09ee-4909-977b-2d730eb89c95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:43.669848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af5606-09ee-4909-977b-2d730eb89c95 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:43.675393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 355/718] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:05:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:43.682028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:43.799494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74835fe-2323-40ee-bb77-c712e04b7bd0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:43.802028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:43.865075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac173ac-fc4e-46b9-8e1a-0f79272f4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:43.866324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac173ac-fc4e-46b9-8e1a-0f79272f4cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:43.866825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac173ac-fc4e-46b9-8e1a-0f79272f4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:43.867322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac173ac-fc4e-46b9-8e1a-0f79272f4cb6 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:43.868451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac173ac-fc4e-46b9-8e1a-0f79272f4cb6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:43.868968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac173ac-fc4e-46b9-8e1a-0f79272f4cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:43.871501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac173ac-fc4e-46b9-8e1a-0f79272f4cb6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:43.885480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 364/719] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:05:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:44.232554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593516-ed23-4d97-9455-d2f951d5e8b7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:44.233098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593516-ed23-4d97-9455-d2f951d5e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:44.233479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593516-ed23-4d97-9455-d2f951d5e8b7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:44.235176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 356/720] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:44.240791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1995165-2d56-44de-b754-9278ac2dca90 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:44.241195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1995165-2d56-44de-b754-9278ac2dca90 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:44.241564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1995165-2d56-44de-b754-9278ac2dca90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:44.261363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f1995165-2d56-44de-b754-9278ac2dca90 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:44.262679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f1995165-2d56-44de-b754-9278ac2dca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:44.336489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 365/721] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:44.737745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14593516-ed23-4d97-9455-d2f951d5e8b7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:44.825709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:44.864150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14593516-ed23-4d97-9455-d2f951d5e8b7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:44.905299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472cfd2-e075-416a-8434-241f5d95e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:44.905940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472cfd2-e075-416a-8434-241f5d95e2dd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:44.906392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472cfd2-e075-416a-8434-241f5d95e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:44.906665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472cfd2-e075-416a-8434-241f5d95e2dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:44.917508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472cfd2-e075-416a-8434-241f5d95e2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2472cfd2-e075-416a-8434-241f5d95e2dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:44.922887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:44.923894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:44.925280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 357/722] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:44.932674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1995165-2d56-44de-b754-9278ac2dca90 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:45.065887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:45.069671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1995165-2d56-44de-b754-9278ac2dca90 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:45.104222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93293a-49bf-44d8-a3bb-62a96ba8d4a6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.104593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93293a-49bf-44d8-a3bb-62a96ba8d4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.104880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93293a-49bf-44d8-a3bb-62a96ba8d4a6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.106967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 358/723] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:45.112165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f481297-6065-4b7c-9d0f-dbdf859c90a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:45.112788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f481297-6065-4b7c-9d0f-dbdf859c90a1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.113065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f481297-6065-4b7c-9d0f-dbdf859c90a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.113337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f481297-6065-4b7c-9d0f-dbdf859c90a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.113826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6fde5-7374-4b45-91b6-c1ac8a59819f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.114015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f481297-6065-4b7c-9d0f-dbdf859c90a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:45.114322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f481297-6065-4b7c-9d0f-dbdf859c90a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:45.114690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6fde5-7374-4b45-91b6-c1ac8a59819f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.115151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6fde5-7374-4b45-91b6-c1ac8a59819f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.115793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f481297-6065-4b7c-9d0f-dbdf859c90a1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:45.125374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 366/724] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:45.130178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2680a2-c43b-4ede-af9f-46c966f37860 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.130459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2680a2-c43b-4ede-af9f-46c966f37860 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.130735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2680a2-c43b-4ede-af9f-46c966f37860 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.132228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 367/725] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:05:45.136651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2680a2-c43b-4ede-af9f-46c966f37860 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:45.146900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0ee6fde5-7374-4b45-91b6-c1ac8a59819f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:45.148923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0ee6fde5-7374-4b45-91b6-c1ac8a59819f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:45.235387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 359/726] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:45.237097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b469d50-a619-4a37-8843-1f58b6064bf0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.237570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b469d50-a619-4a37-8843-1f58b6064bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.238045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b469d50-a619-4a37-8843-1f58b6064bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.271093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0b469d50-a619-4a37-8843-1f58b6064bf0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:45.274083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0b469d50-a619-4a37-8843-1f58b6064bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:45.279546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2680a2-c43b-4ede-af9f-46c966f37860 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:45.340487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:45.341515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.342005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.342460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.365757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:45.367462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:45.391612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:45.397705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:45.398679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:45.399750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:45.399750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:45.399750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:45.407225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 368/727] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:45.414625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:45.419186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 360/728] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:45.440550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:45.562358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62790013-71a3-4119-acaa-b359f3bae833 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:45.588847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b469d50-a619-4a37-8843-1f58b6064bf0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:45.623964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace110-2e3c-46fc-b2db-d62018fcb04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:45.624917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace110-2e3c-46fc-b2db-d62018fcb04f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.625412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace110-2e3c-46fc-b2db-d62018fcb04f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.625880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace110-2e3c-46fc-b2db-d62018fcb04f None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.626729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cace110-2e3c-46fc-b2db-d62018fcb04f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:45.627224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cace110-2e3c-46fc-b2db-d62018fcb04f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:45.629007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cace110-2e3c-46fc-b2db-d62018fcb04f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:45.651358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 369/729] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:45.652647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8312f-51ce-417c-a871-98e846b029aa None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.657274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8312f-51ce-417c-a871-98e846b029aa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.657274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8312f-51ce-417c-a871-98e846b029aa None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.657274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 370/730] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:45.659720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8312f-51ce-417c-a871-98e846b029aa None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:45.747042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b469d50-a619-4a37-8843-1f58b6064bf0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:45.774577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:45.793766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5547f-1f0c-4de2-a9dd-a234260758c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:45.794583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5547f-1f0c-4de2-a9dd-a234260758c4 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.794858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5547f-1f0c-4de2-a9dd-a234260758c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.795126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5547f-1f0c-4de2-a9dd-a234260758c4 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.796274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5547f-1f0c-4de2-a9dd-a234260758c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:45.797870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5547f-1f0c-4de2-a9dd-a234260758c4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:45.821835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 361/731] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:05:45 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:05:45.824176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f69a9b-a4a7-4dfe-8e8a-27436cf9394c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.824796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f69a9b-a4a7-4dfe-8e8a-27436cf9394c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.825151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f69a9b-a4a7-4dfe-8e8a-27436cf9394c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.846920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8312f-51ce-417c-a871-98e846b029aa None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:45.856626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b1f69a9b-a4a7-4dfe-8e8a-27436cf9394c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:45.858626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b1f69a9b-a4a7-4dfe-8e8a-27436cf9394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:45.899340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:45.900279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:45.900760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:45.901169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:45.913025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:45.915541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:45.916900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:45.933693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:45.935464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:45.936644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:45.942151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 371/732] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:45.942151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:45.959217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 362/733] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:45.960533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f69a9b-a4a7-4dfe-8e8a-27436cf9394c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.049886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f94b8-c86f-44c0-9dcb-b5359ddd91ea None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.078185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:46.080119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f69a9b-a4a7-4dfe-8e8a-27436cf9394c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.086993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051db585-f307-46ad-9e3c-aa61205e72e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.087822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051db585-f307-46ad-9e3c-aa61205e72e9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.088211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051db585-f307-46ad-9e3c-aa61205e72e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.088729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051db585-f307-46ad-9e3c-aa61205e72e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.101218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051db585-f307-46ad-9e3c-aa61205e72e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-051db585-f307-46ad-9e3c-aa61205e72e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:46.108041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:46.110140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:46.111990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:46.116163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 372/734] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:46.117551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051db585-f307-46ad-9e3c-aa61205e72e9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.131910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.132967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.133378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.133783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.134700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:46.135156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:46.137066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:46.163158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 363/735] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 31 12:05:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:46.169484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.209898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:46.229234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051db585-f307-46ad-9e3c-aa61205e72e9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.265686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.266379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.266728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.267075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.267898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:46.268195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:46.269758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:46.279650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 373/736] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:46.285255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.286008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403a9e4-0781-4e0e-9c59-1bd3e522bd82 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.324578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c55b2-e504-4fa7-90fb-31ee5c595fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.325637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c55b2-e504-4fa7-90fb-31ee5c595fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.326088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c55b2-e504-4fa7-90fb-31ee5c595fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.326587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c55b2-e504-4fa7-90fb-31ee5c595fd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.341086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:46.399344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161f7a4-dd0a-4a6c-b0e9-87e1fddaf690 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.411896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c55b2-e504-4fa7-90fb-31ee5c595fd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:46.413770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c55b2-e504-4fa7-90fb-31ee5c595fd6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:46.438172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8bb35-2dac-4e3a-bc36-ef0d453f1728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.439048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8bb35-2dac-4e3a-bc36-ef0d453f1728 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.439354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8bb35-2dac-4e3a-bc36-ef0d453f1728 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.439700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8bb35-2dac-4e3a-bc36-ef0d453f1728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.450837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:46.452845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8bb35-2dac-4e3a-bc36-ef0d453f1728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:46.454333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8bb35-2dac-4e3a-bc36-ef0d453f1728 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b4f8bb35-2dac-4e3a-bc36-ef0d453f1728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:46.470796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:46.471938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:46.473169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:46.473169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:46.473169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:46.473169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 374/737] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:46.478411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b497ab0-f34b-4b96-9eda-daba255deb49 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.478697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b497ab0-f34b-4b96-9eda-daba255deb49 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.478975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b497ab0-f34b-4b96-9eda-daba255deb49 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.480593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 375/738] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:46.481326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b497ab0-f34b-4b96-9eda-daba255deb49 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.522056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 364/739] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 12:05:46.527960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9ae8-0714-48ad-8170-2d6e3b5ee115 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.528415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9ae8-0714-48ad-8170-2d6e3b5ee115 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.528827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9ae8-0714-48ad-8170-2d6e3b5ee115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.547517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2bcd9ae8-0714-48ad-8170-2d6e3b5ee115 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:46.548925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2bcd9ae8-0714-48ad-8170-2d6e3b5ee115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:46.588229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b497ab0-f34b-4b96-9eda-daba255deb49 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.596100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:46.621033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 365/740] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 12:05:46.621982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd9ae8-0714-48ad-8170-2d6e3b5ee115 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.629268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.630148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.631065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.631690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.632552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:46.633379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:46.635635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:46.650913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 376/741] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:46.652196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.707983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:46.745561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd9ae8-0714-48ad-8170-2d6e3b5ee115 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.774927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556e99-bc3f-4f66-bd1c-9eead803d418 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.781614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.782321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.782662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.782995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.794399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:46.796216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:46.812189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:46.813223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:46.814140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:46.814140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:46.814140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:46.816820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e79fd-5639-46c4-851f-10cc6feb375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.817573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e79fd-5639-46c4-851f-10cc6feb375f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.818140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 366/742] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:46.818307 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e79fd-5639-46c4-851f-10cc6feb375f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.818491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e79fd-5639-46c4-851f-10cc6feb375f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.819190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.831325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e79fd-5639-46c4-851f-10cc6feb375f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-0d4e79fd-5639-46c4-851f-10cc6feb375f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:46.837605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:46.839633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:46.841857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:46.851277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 377/743] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:46.851277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e79fd-5639-46c4-851f-10cc6feb375f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:46.880011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:46.928319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbf618-63a0-43e2-bcba-ff661754d615 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.953946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:46.964547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e79fd-5639-46c4-851f-10cc6feb375f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:46.965043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:46.965591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:46.965952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:46.966384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:46.974973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:46.976495 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:46.988105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:46.989090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 367/744] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:46.990034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.007305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.008347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.008822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.009282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.010059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:47.010500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.012048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.024834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 378/745] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:47.025865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.104817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc05ec-caaf-4938-ae51-8a9a59a91c07 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.112272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:47.133836 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e25c1-7c44-44cc-a099-155931f75e66 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.142942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.143667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.143954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.144224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.144941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:47.145251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.146640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.156605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 368/746] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:47.157440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.174853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56642792-81f1-44ad-aca6-55ba17a3e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.175782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642792-81f1-44ad-aca6-55ba17a3e318 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.176070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642792-81f1-44ad-aca6-55ba17a3e318 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.176336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642792-81f1-44ad-aca6-55ba17a3e318 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.176998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56642792-81f1-44ad-aca6-55ba17a3e318 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.178373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56642792-81f1-44ad-aca6-55ba17a3e318 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.195000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 379/747] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:05:47 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:47.200410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56642792-81f1-44ad-aca6-55ba17a3e318 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.209650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:47.288787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c72fe-d7a9-45d8-a597-61adfb37609f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.337104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.337789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.338068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.338338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.339167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:47.339427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.343039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.354640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56642792-81f1-44ad-aca6-55ba17a3e318 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.369130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 369/748] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:47.371744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.396111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.396776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.397061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.397357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.405529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.407079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:47.422747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:47.424391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 380/749] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:47.426078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.436475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:47.520101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:47.527161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348f5f-da64-441d-a725-4cfac9e2cae0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.562881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.563526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.563804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.564095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.565296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b36a-876f-447b-bc9e-e37556673b47 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.575107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.576673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:47.592520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:47.593733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:47.594744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:47.594744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:47.594744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:47.598762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 370/750] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:47.599737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.607693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.608572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.608995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.609524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.621899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.624247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:47.641364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:47.642618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:47.643611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:47.643611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:47.643611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:47.651181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 381/751] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:47.652364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.707367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366fd35-a932-42a0-9394-6af4dcdf3ba9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.714639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:47.744902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.745669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.745949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.746313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.747560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:47.747866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.749814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.767438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 371/752] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:47.768567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.775709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d13848-f452-41c9-836a-01d708f381b0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.785952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:47.812214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.812879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.813219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.813440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.814095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:47.814402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.815873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.828891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 382/753] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:47.829596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:47.922791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6c858-1733-4615-bde1-6006e3fb9cac None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.949797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107f91-7f72-4caa-9166-5350c810e916 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:47.961834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:47.962452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:47.962726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:47.962991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:47.963665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:47.964258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:47.965435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:47.969035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:47.981413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 372/754] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:47.982613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:48.006426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:48.007343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.007759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.008164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.009123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=981e9da7f2774c90a667fab35ecab64b) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:48.010049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:48.011936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:48.019572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 383/755] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/role_assignments?user.id=981e9da7f2774c90a667fab35ecab64b&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:48.021139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:48.068479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:48.155796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf27c9-f8a3-4fb4-b617-ce4ec4183465 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:48.179617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a39a0-2f00-48c0-ad94-36eaad584926 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:48.224106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc7e4b-7875-4e8e-9b85-369da194ed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:48.225641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc7e4b-7875-4e8e-9b85-369da194ed50 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.226066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc7e4b-7875-4e8e-9b85-369da194ed50 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.226492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc7e4b-7875-4e8e-9b85-369da194ed50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.235533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc7e4b-7875-4e8e-9b85-369da194ed50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:48.237439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc7e4b-7875-4e8e-9b85-369da194ed50 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:48.238287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:48.238939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.239263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.239562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.250362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-50fc7e4b-7875-4e8e-9b85-369da194ed50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:48.252010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:48.253018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:48.253905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 384/756] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:48.254384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bd076-8661-4894-9a22-f36d344b1477 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.254576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bd076-8661-4894-9a22-f36d344b1477 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.254819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bd076-8661-4894-9a22-f36d344b1477 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.255821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:48.256438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 385/757] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:48.262416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065818ef-f34b-4db7-98ac-4744b337362c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.262987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065818ef-f34b-4db7-98ac-4744b337362c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.262987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065818ef-f34b-4db7-98ac-4744b337362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:48.275867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:48.276887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:05:48.277858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:48.281456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 373/758] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:05:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:48.282380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:48.292333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-065818ef-f34b-4db7-98ac-4744b337362c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:48.294525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-065818ef-f34b-4db7-98ac-4744b337362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:48.335211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:48.391262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 386/759] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:05:48.391262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065818ef-f34b-4db7-98ac-4744b337362c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:48.422419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33940d98-990e-42c8-8fe0-62910189532e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:48.475630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:48.489349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:48.490069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.490352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.490627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.491393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:48.491808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:48.493338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:48.507404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 374/760] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:48.514090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:48.557085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065818ef-f34b-4db7-98ac-4744b337362c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:48.629170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:48.629706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.629981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.630252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.633337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:48.633665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:48.637914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:48.678949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 387/761] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 31 12:05:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:48.683033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:48.721994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a767e8-f0e3-4118-9943-c306fcb70a0e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:48.735590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:48.761157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbe660-1a34-4b16-b16c-2d216e653b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:48.761835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbe660-1a34-4b16-b16c-2d216e653b32 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.762171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbe660-1a34-4b16-b16c-2d216e653b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.762449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbe660-1a34-4b16-b16c-2d216e653b32 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/0b113316bc3348b5a12c1d7b616bf4c9/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.764681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbe660-1a34-4b16-b16c-2d216e653b32 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=0b113316bc3348b5a12c1d7b616bf4c9, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:48.766137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbe660-1a34-4b16-b16c-2d216e653b32 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:48.786089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 375/762] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:05:48 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/0b113316bc3348b5a12c1d7b616bf4c9/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:05:48.787282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbe660-1a34-4b16-b16c-2d216e653b32 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:48.850941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:48.883349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba234d17-5350-4b3d-bec2-5173514880ed None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:48.920593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbe660-1a34-4b16-b16c-2d216e653b32 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:48.963294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:48.964313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.964593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.965195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:48.965832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:48.968177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:48.979377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:48.980447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:48.981352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 376/763] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 31 12:05:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:48.988589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:48.994142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393f771-c8b7-453d-8c15-08ff9d6b42fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:48.994809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393f771-c8b7-453d-8c15-08ff9d6b42fd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:48.995047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393f771-c8b7-453d-8c15-08ff9d6b42fd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:48.995350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393f771-c8b7-453d-8c15-08ff9d6b42fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.105634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393f771-c8b7-453d-8c15-08ff9d6b42fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:49.107779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393f771-c8b7-453d-8c15-08ff9d6b42fd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:49.140053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:49.154906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74bd3a-1611-4691-980c-46dc87d66489 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:49.172995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:49.223280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:49.224381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:49.224863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:49.225327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.226178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:49.226683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:49.228498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:49.234686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 377/764] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 31 12:05:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:49.240815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:49.283301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 388/765] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:05:49.289283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393f771-c8b7-453d-8c15-08ff9d6b42fd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:49.419655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393f771-c8b7-453d-8c15-08ff9d6b42fd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:49.449504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:49.449825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f70e-5d0c-493e-8bb6-0d9d6a6edd61 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:49.460434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c6c25-a1ba-4cdf-92d0-a324fb933360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:49.461437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6c25-a1ba-4cdf-92d0-a324fb933360 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:49.461998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6c25-a1ba-4cdf-92d0-a324fb933360 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:49.462471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6c25-a1ba-4cdf-92d0-a324fb933360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.474527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c6c25-a1ba-4cdf-92d0-a324fb933360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:49.476423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c6c25-a1ba-4cdf-92d0-a324fb933360 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:49.489280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158acd1-7d49-42f3-88a4-fd8da250a341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:49.489886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158acd1-7d49-42f3-88a4-fd8da250a341 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:49.490160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158acd1-7d49-42f3-88a4-fd8da250a341 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:49.490427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158acd1-7d49-42f3-88a4-fd8da250a341 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.491047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158acd1-7d49-42f3-88a4-fd8da250a341 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:49.491405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158acd1-7d49-42f3-88a4-fd8da250a341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:49.492781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158acd1-7d49-42f3-88a4-fd8da250a341 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f59c6c25-a1ba-4cdf-92d0-a324fb933360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:49.495524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:49.496463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:49.497576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:49.497576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:49.497576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:49.499126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 378/766] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 31 12:05:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:49.506813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c92d8-f902-40ee-9f66-bf5e386f5e3a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:49.506813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c92d8-f902-40ee-9f66-bf5e386f5e3a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:49.506813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c92d8-f902-40ee-9f66-bf5e386f5e3a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.506813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 379/767] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:49.507116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 389/768] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:49.507217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c92d8-f902-40ee-9f66-bf5e386f5e3a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:49.509830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19082b-c920-438c-884e-f10f4671b008 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:49.510914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19082b-c920-438c-884e-f10f4671b008 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:49.510914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19082b-c920-438c-884e-f10f4671b008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.536883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ee19082b-c920-438c-884e-f10f4671b008 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:49.539155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ee19082b-c920-438c-884e-f10f4671b008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:49.560047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:49.639494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 390/769] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:05:49.640430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19082b-c920-438c-884e-f10f4671b008 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:49.641554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c92d8-f902-40ee-9f66-bf5e386f5e3a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:49.708415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:49.708966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:49.709244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:49.709519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.710226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] RBAC: Adding query filter params (interface=public, service_id=1dc428b17e664575ad781f245f7d8733, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:49.710535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:49.711973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:49.737755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 380/770] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 31 12:05:49 2022] GET /identity/v3/endpoints?service_id=1dc428b17e664575ad781f245f7d8733&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:49.738720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:49.838883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19082b-c920-438c-884e-f10f4671b008 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:49.855174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:49.928775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:49.929391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:49.929665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:49.929935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.930588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:49.930891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:49.937133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:49.950715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc37809-e4aa-4c1c-bbce-e4965e254d2c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:49.954040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:49.959004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 391/771] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:49.964004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:49.992280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc916ae-c493-4700-b445-9884379fcb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:49.992903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc916ae-c493-4700-b445-9884379fcb6b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:49.993213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc916ae-c493-4700-b445-9884379fcb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:49.993443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc916ae-c493-4700-b445-9884379fcb6b None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:49.994055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc916ae-c493-4700-b445-9884379fcb6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:49.995474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc916ae-c493-4700-b445-9884379fcb6b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:50.002176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 381/772] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:05:49 2022] GET /identity/v3/services => generated 772 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:50.003047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc916ae-c493-4700-b445-9884379fcb6b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:50.090339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea7145-cebb-4b9c-ae25-5ef951de4be3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:50.129536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:50.130663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.131145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.131669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.133659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc916ae-c493-4700-b445-9884379fcb6b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:50.140598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:50.142376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:50.154984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:50.156028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:50.156905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:50.156905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:50.156905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:50.156905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:50.156905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:50.156905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:50.156905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:50.156905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:50.159380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 392/773] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:50.168406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:50.181634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e4ad5-ec59-4e78-8aec-5494e9d35cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:50.182353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e4ad5-ec59-4e78-8aec-5494e9d35cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.182633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e4ad5-ec59-4e78-8aec-5494e9d35cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.182913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e4ad5-ec59-4e78-8aec-5494e9d35cd2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.183596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e4ad5-ec59-4e78-8aec-5494e9d35cd2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:50.185427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e4ad5-ec59-4e78-8aec-5494e9d35cd2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9c5e4ad5-ec59-4e78-8aec-5494e9d35cd2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:50.195481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:50.196586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:50.197424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 382/774] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:50 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:50.203302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27616749-1a86-4cae-9659-caad49ba0604 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.203302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27616749-1a86-4cae-9659-caad49ba0604 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.203302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27616749-1a86-4cae-9659-caad49ba0604 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.203302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 383/775] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:50.207017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27616749-1a86-4cae-9659-caad49ba0604 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:50.221286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:50.260251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:50.327055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18780e-cd19-45b0-ab9d-501c6526ed30 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:50.347915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27616749-1a86-4cae-9659-caad49ba0604 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:50.376070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:50.376712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.376991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.377263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.378335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:50.378934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:50.381221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:50.393000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00781e-b017-4161-b9df-3c8fc208aafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:50.393975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00781e-b017-4161-b9df-3c8fc208aafd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.394399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00781e-b017-4161-b9df-3c8fc208aafd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.394814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00781e-b017-4161-b9df-3c8fc208aafd None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.395628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00781e-b017-4161-b9df-3c8fc208aafd None admin] RBAC: Adding query filter params (name=network) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:50.396109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00781e-b017-4161-b9df-3c8fc208aafd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:50.397752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00781e-b017-4161-b9df-3c8fc208aafd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:50.405079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 384/776] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:05:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:50.407305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d62b5-eb57-4fbb-998d-b66dee4faae6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.407842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d62b5-eb57-4fbb-998d-b66dee4faae6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.407980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 393/777] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:50.408382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d62b5-eb57-4fbb-998d-b66dee4faae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.413331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:50.431486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9f2d62b5-eb57-4fbb-998d-b66dee4faae6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:50.432869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9f2d62b5-eb57-4fbb-998d-b66dee4faae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:50.529074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:50.530751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 385/778] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:05:50.532725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d62b5-eb57-4fbb-998d-b66dee4faae6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:50.544527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43028955-0fac-49c0-8431-0f99949c9f99 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:50.572659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:50.586316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:50.586933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.587212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.587636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.588291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] RBAC: Adding query filter params (type=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:50.588591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:50.589987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:50.596170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 394/779] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:05:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:50.604616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:50.667945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d62b5-eb57-4fbb-998d-b66dee4faae6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:50.723128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:50.723814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.724089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.724356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.725083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=ce1444d8c66045389eac37f8a7309378) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:50.725391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:50.726847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:50.742392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 386/780] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:50 2022] GET /identity/v3/role_assignments?user.id=ce1444d8c66045389eac37f8a7309378&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:50.758242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955409f-d089-4772-b925-8116cbff5054 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:50.806870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:50.807457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.807788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.808066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.808792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] RBAC: Adding query filter params (interface=public, service_id=4cc1ee88f6854f0abd11dac5862412b2, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:50.809102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:50.811017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:50.821005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 395/781] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 31 12:05:50 2022] GET /identity/v3/endpoints?service_id=4cc1ee88f6854f0abd11dac5862412b2&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:50.826600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:50.951901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:50.957107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891a3a2-8260-4393-8893-b8b578680cfb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:50.996405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c91fe-2ae5-4a2f-a9e5-3eb918fb62df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:50.996648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:50.997715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c91fe-2ae5-4a2f-a9e5-3eb918fb62df None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:50.998397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c91fe-2ae5-4a2f-a9e5-3eb918fb62df None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:50.998675 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c91fe-2ae5-4a2f-a9e5-3eb918fb62df None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:50.999327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c91fe-2ae5-4a2f-a9e5-3eb918fb62df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:51.000802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c91fe-2ae5-4a2f-a9e5-3eb918fb62df None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:51.006626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 387/782] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:05:50 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:51.036378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:51.165536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a886cb-aa25-4750-b61b-530e1d795dce None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:51.208160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa7c4-5090-44c6-8137-ef19adf024df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:51.209104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa7c4-5090-44c6-8137-ef19adf024df None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:51.209740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa7c4-5090-44c6-8137-ef19adf024df None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:51.210349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa7c4-5090-44c6-8137-ef19adf024df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:51.229679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc2bf-82d8-498d-89c0-846630f47180 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:51.230086 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc2bf-82d8-498d-89c0-846630f47180 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:51.230482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc2bf-82d8-498d-89c0-846630f47180 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:51.232761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 388/783] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:51.233281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa7c4-5090-44c6-8137-ef19adf024df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:51.239726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455693e-fb12-459f-8e64-6e5fcf352589 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:51.240106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455693e-fb12-459f-8e64-6e5fcf352589 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:51.240377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455693e-fb12-459f-8e64-6e5fcf352589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:51.271950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c455693e-fb12-459f-8e64-6e5fcf352589 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:51.273290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c455693e-fb12-459f-8e64-6e5fcf352589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:51.368943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 389/784] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f13aa7c4-5090-44c6-8137-ef19adf024df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:51.380356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:51.381376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:51.382231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:51.386439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 396/785] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:51.388281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455693e-fb12-459f-8e64-6e5fcf352589 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:51.499825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:51.526111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455693e-fb12-459f-8e64-6e5fcf352589 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:51.580192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:51.580811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:51.581083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:51.581351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:51.582016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:51.582312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:51.583945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:51.593701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 390/786] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:51.599830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa7c4-5090-44c6-8137-ef19adf024df None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:51.726437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:51.741883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:51.766624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa7c4-5090-44c6-8137-ef19adf024df None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:51.830475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:51.832259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:51.832540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:51.832811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:51.853551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:51.855432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:51.884605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:51.885630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:51.886518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:51.886518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:51.886518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:51.886518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 397/787] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:51.898206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:51.956805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53348cb8-8e90-4521-99ce-22eb3ce2dc28 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:52.038501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c46f3d-8f27-49fd-8a37-a0671a8f774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:52.039076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46f3d-8f27-49fd-8a37-a0671a8f774d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.039384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46f3d-8f27-49fd-8a37-a0671a8f774d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.039659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46f3d-8f27-49fd-8a37-a0671a8f774d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.072724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c46f3d-8f27-49fd-8a37-a0671a8f774d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e0c46f3d-8f27-49fd-8a37-a0671a8f774d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:52.080852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:52.083094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:52.085110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:52.091773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 391/788] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:52.094685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09dc5-95ca-4180-afa4-ba84489af7eb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:52.098323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c46f3d-8f27-49fd-8a37-a0671a8f774d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:52.112835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:52.129331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:52.139862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:52.140494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.140776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.141050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.141713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:52.142020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:52.143671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:52.161169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 398/789] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:52.163500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:52.302001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311eb33-51e3-49b0-b457-cfaf0e3d5fd2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:52.349268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c46f3d-8f27-49fd-8a37-a0671a8f774d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:52.387756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:52.388790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.389075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.389351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.390164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:52.390512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:52.392013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:52.403560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 392/790] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:52.409176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:52.411775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:52.414648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.414936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.415207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.435225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:52.436984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:52.462281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:52.464497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:52.466743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:52.473668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 399/791] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:52.474735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:52.534464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:52.554077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:52.588584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b63cb-0dbd-4c2d-b2e2-2227fe3d9613 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:52.638910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b6768-c8ff-4b28-b472-0cb14ed021ec None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:52.640398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c80b-4d98-4354-910f-f4892a4e8c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:52.641201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c80b-4d98-4354-910f-f4892a4e8c9c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.641591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c80b-4d98-4354-910f-f4892a4e8c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.642106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c80b-4d98-4354-910f-f4892a4e8c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.657707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c80b-4d98-4354-910f-f4892a4e8c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:52.659653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c80b-4d98-4354-910f-f4892a4e8c9c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b393c80b-4d98-4354-910f-f4892a4e8c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:52.682166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:52.683201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:52.684137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:52.684137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:52.684137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:52.686958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 393/792] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:52.688542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3563c5-f8d9-46b1-8198-d9f6214ef599 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.689035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3563c5-f8d9-46b1-8198-d9f6214ef599 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.689526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3563c5-f8d9-46b1-8198-d9f6214ef599 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.691334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 394/793] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:52.693424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3563c5-f8d9-46b1-8198-d9f6214ef599 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:52.694331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf1d14-91b4-4093-a9e4-eedb7b64318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:52.695611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf1d14-91b4-4093-a9e4-eedb7b64318e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.696164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf1d14-91b4-4093-a9e4-eedb7b64318e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.696709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf1d14-91b4-4093-a9e4-eedb7b64318e None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.698191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf1d14-91b4-4093-a9e4-eedb7b64318e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:52.698830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf1d14-91b4-4093-a9e4-eedb7b64318e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:52.701817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf1d14-91b4-4093-a9e4-eedb7b64318e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:52.735470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 400/794] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 31 12:05:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:52.738133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d658ee9-9641-483c-8aec-c826840ef07d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.738748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d658ee9-9641-483c-8aec-c826840ef07d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.739373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d658ee9-9641-483c-8aec-c826840ef07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.763364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9d658ee9-9641-483c-8aec-c826840ef07d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:52.764673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9d658ee9-9641-483c-8aec-c826840ef07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:52.829975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3563c5-f8d9-46b1-8198-d9f6214ef599 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:52.844978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 401/795] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:05:52.846137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d658ee9-9641-483c-8aec-c826840ef07d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:52.853481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:52.884885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:52.885830 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:52.886336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:52.886787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:52.887751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:52.888397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:52.895621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:52.896762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:52.913956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 395/796] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:52.929117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:52.975998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d658ee9-9641-483c-8aec-c826840ef07d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.021050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:53.021900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.022274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.022641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.035432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:53.037096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:53.054835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:53.056106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:53.057226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:53.057226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:53.057226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:53.063459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 402/797] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:53.065212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:53.122341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18825d7-58bd-492b-80e0-cfc1ad5c3a56 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.165068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:53.166094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.166565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.167103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.172992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:53.178376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:53.181067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:53.188327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f33f6-6b6d-4701-9b98-5f4ef627ce7e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:53.195455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:53.196485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:05:53.197442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:53.200424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 396/798] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:05:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:53.206027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:53.230638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:53.231503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.232133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.232594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.233348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:53.233726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:53.235346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:53.254713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 403/799] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:53.256090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:53.259691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:53.335672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69793-c913-41bb-9ea9-c54ddb7da40c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.378412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:53.378614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0c9ce-d036-4c58-8487-77a8e303ceb0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.379036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.379312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.379541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.380150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:53.382112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:53.391559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:53.392652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:53.393514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 397/800] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 31 12:05:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:53.394363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:53.429221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44337bed-a88c-4d07-9981-e634591ecf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:53.429915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337bed-a88c-4d07-9981-e634591ecf54 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.430229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337bed-a88c-4d07-9981-e634591ecf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.430473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337bed-a88c-4d07-9981-e634591ecf54 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.431357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337bed-a88c-4d07-9981-e634591ecf54 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:53.431636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337bed-a88c-4d07-9981-e634591ecf54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:53.433219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337bed-a88c-4d07-9981-e634591ecf54 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:53.461209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 404/801] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:05:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:53.461209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44c6e0-90e8-40bf-863a-e6568465456b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.461209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44c6e0-90e8-40bf-863a-e6568465456b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.461209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44c6e0-90e8-40bf-863a-e6568465456b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.461209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 405/802] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:53.461679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44c6e0-90e8-40bf-863a-e6568465456b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:53.473184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:53.586644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44c6e0-90e8-40bf-863a-e6568465456b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.624923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:53.625564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.625840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.626104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.626773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:53.627070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:53.628602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:53.641569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 406/803] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 31 12:05:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:53.641569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:53.652206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6ddee-032e-45d9-8998-6a63b0ed965b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.654091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:53.698538 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb769c0-f3ec-46d7-96aa-77b71e1837d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:53.699179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb769c0-f3ec-46d7-96aa-77b71e1837d9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.699488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb769c0-f3ec-46d7-96aa-77b71e1837d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.699758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb769c0-f3ec-46d7-96aa-77b71e1837d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.707690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb769c0-f3ec-46d7-96aa-77b71e1837d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:53.709240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb769c0-f3ec-46d7-96aa-77b71e1837d9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-0bb769c0-f3ec-46d7-96aa-77b71e1837d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:53.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:53.723511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 398/804] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8037d-7630-4c38-be29-a97dc102fb42 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.725273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8037d-7630-4c38-be29-a97dc102fb42 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.726449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8037d-7630-4c38-be29-a97dc102fb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.733302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:53.745947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4bf8037d-7630-4c38-be29-a97dc102fb42 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:53.747275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4bf8037d-7630-4c38-be29-a97dc102fb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:53.771189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42da7d5-4534-431d-9bab-dc63fa45d2a1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.812721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e506c-c945-488b-ad85-63672507ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:53.813470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e506c-c945-488b-ad85-63672507ac82 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:53.813859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e506c-c945-488b-ad85-63672507ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:53.814302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e506c-c945-488b-ad85-63672507ac82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:53.823910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 399/805] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:05:53.825510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8037d-7630-4c38-be29-a97dc102fb42 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:53.898934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e506c-c945-488b-ad85-63672507ac82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:53.901367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e506c-c945-488b-ad85-63672507ac82 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:53.979886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:53.993980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8037d-7630-4c38-be29-a97dc102fb42 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:53.998021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:54.057198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:54.057900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:54.058219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:54.058531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:54.059367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:54.059817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:54.061519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:54.066814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 407/806] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:05:54.068745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 400/807] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 31 12:05:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:54.070457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:54.076006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9264f-8592-47c8-8f2b-f97204f6cb7a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:54.076473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9264f-8592-47c8-8f2b-f97204f6cb7a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:54.076961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9264f-8592-47c8-8f2b-f97204f6cb7a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:54.079896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 408/808] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:54 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:54.081265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9264f-8592-47c8-8f2b-f97204f6cb7a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:54.239871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc4b78-34ba-4922-a974-b6a9552e173c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:54.259384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9264f-8592-47c8-8f2b-f97204f6cb7a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:54.291574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:54.292439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:54.292857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:54.293261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:54.294397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:54.294397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:54.296731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:54.307167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485c177-d1b2-489d-a483-42d8fb9d1b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:54.307943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485c177-d1b2-489d-a483-42d8fb9d1b9d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:54.308370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485c177-d1b2-489d-a483-42d8fb9d1b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:54.308798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485c177-d1b2-489d-a483-42d8fb9d1b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:54.312963 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 401/809] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:54.314041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:54.325634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485c177-d1b2-489d-a483-42d8fb9d1b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:54.326908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485c177-d1b2-489d-a483-42d8fb9d1b9d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:54.336013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3485c177-d1b2-489d-a483-42d8fb9d1b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:54.352115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:54.353950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:54.355525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:54.355525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:54.355525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:54.355525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 409/810] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:54.356150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce097f-c590-4b11-b6f4-1e4c1bd21389 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:54.356150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce097f-c590-4b11-b6f4-1e4c1bd21389 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:54.356761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce097f-c590-4b11-b6f4-1e4c1bd21389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:54.378940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-63ce097f-c590-4b11-b6f4-1e4c1bd21389 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:54.380180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-63ce097f-c590-4b11-b6f4-1e4c1bd21389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:54.400101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:54.449662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af22c6-c31a-452c-9d18-c9f4447f4707 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:54.462139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 410/811] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:05:54.463738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce097f-c590-4b11-b6f4-1e4c1bd21389 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:54.506592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7335e80-dbd7-4005-b4b6-1389c4ce5ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:54.507464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7335e80-dbd7-4005-b4b6-1389c4ce5ae4 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:54.507923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7335e80-dbd7-4005-b4b6-1389c4ce5ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:54.508392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7335e80-dbd7-4005-b4b6-1389c4ce5ae4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:54.509187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7335e80-dbd7-4005-b4b6-1389c4ce5ae4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:54.510940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7335e80-dbd7-4005-b4b6-1389c4ce5ae4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:54.547736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 402/812] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:05:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:54.548800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7335e80-dbd7-4005-b4b6-1389c4ce5ae4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:54.688420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce097f-c590-4b11-b6f4-1e4c1bd21389 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:54.701875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:54.765434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438de2f-69c8-490d-9a0d-42227c96e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:54.766465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438de2f-69c8-490d-9a0d-42227c96e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:54.766832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438de2f-69c8-490d-9a0d-42227c96e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:54.768199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438de2f-69c8-490d-9a0d-42227c96e3f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:54.774074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7335e80-dbd7-4005-b4b6-1389c4ce5ae4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:54.791886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438de2f-69c8-490d-9a0d-42227c96e3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f438de2f-69c8-490d-9a0d-42227c96e3f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:54.800418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:54.801433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:05:54.802282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:54.802961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 411/813] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:54.803722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438de2f-69c8-490d-9a0d-42227c96e3f5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:54.821467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057336-5217-4ae0-8493-99a5024b9837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:54.822516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057336-5217-4ae0-8493-99a5024b9837 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:54.823022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057336-5217-4ae0-8493-99a5024b9837 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:54.823580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057336-5217-4ae0-8493-99a5024b9837 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/981e9da7f2774c90a667fab35ecab64b/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:54.823760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:54.826373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97057336-5217-4ae0-8493-99a5024b9837 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=981e9da7f2774c90a667fab35ecab64b, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:54.827897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97057336-5217-4ae0-8493-99a5024b9837 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:54.846432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 403/814] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:05:54 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/981e9da7f2774c90a667fab35ecab64b/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:05:54.849955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057336-5217-4ae0-8493-99a5024b9837 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:54.973710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057336-5217-4ae0-8493-99a5024b9837 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:54.986057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438de2f-69c8-490d-9a0d-42227c96e3f5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:55.023876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:55.028546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.029842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.030328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.030604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.030994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.031409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.031778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.032122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.032579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:55.032931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:55.033273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.033378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.034835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.035259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.053652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 412/815] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:55.054940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:55.056697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 404/816] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:55.057761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:55.163545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:55.204081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f59d-4fef-412e-ae0d-49932eac6677 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:55.210606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cbc73-4f30-4285-aca6-90cb2e61840e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:55.252537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.253191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.253759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.254035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] PATH_INFO: `/v3/services/network` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.254692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.257006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:55.268187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:55.269312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 12:05:55.270150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:55.270903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.270903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.272141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 413/817] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:55.272285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.272285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.272571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:55.291873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.294608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:55.322460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:55.324736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:55.327000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:55.327000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:55.327000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:55.327000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 405/818] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:55.327000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:55.399314 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:55.437778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc484-8ea0-4cd0-98d7-735cb6c416ec None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:55.483820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:55.492042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.493377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.493844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.494330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.497002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0096922-7c31-4478-a0f0-900a96bd1cfd None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:55.504935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.506843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:55.523661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:55.524829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:55.525896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:55.525896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:55.525896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:55.525896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:55.525896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:55.525896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:55.525896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:55.525896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:55.528515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 414/819] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:55.529915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:55.566111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.566641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.567120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.567622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.568725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:55.569254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.572070 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.584573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 406/820] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:55.584573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:55.675815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2732190-066d-4c1a-b518-3beede94ca4e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:55.720613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.721508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.721932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.723151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.724040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:55.724550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.726549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.747937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 415/821] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:55.749293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:55.766360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7201-6cdc-4abb-b66f-5bd0d67cdfe6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:55.787707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:55.822038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:55.834005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.835505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.835980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.836565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.837893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:55.838535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.841913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.865256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 407/822] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:55.867060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:55.880813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e095c3-558a-44e6-b160-027a07c72c03 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:55.928883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:55.929965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:55.930522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:55.930971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:55.931919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] RBAC: Adding query filter params (type=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:55.932424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:55.934280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:55.941536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 416/823] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:55.942805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:56.071492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786a5e8-a0f8-4bb5-82d7-83d2ed247091 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:56.085285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c9697-eca6-4be8-b15e-b8f145b1e573 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:56.093790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:56.117138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:56.118307 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:56.118730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:56.119142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:56.120216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=0b113316bc3348b5a12c1d7b616bf4c9) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:56.120862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:56.123586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:56.131537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 417/824] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/role_assignments?user.id=0b113316bc3348b5a12c1d7b616bf4c9&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:56.132590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:56.153914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:56.155392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:56.156107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:56.156634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:56.170672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:56.174049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:56.198141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:56.199213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:05:56.200660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:56.205479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 408/825] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:05:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:56.211143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:56.268734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb27e2b-4b44-4bfe-8e76-dd3bb8bdc461 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:56.271686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:56.402099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e9f49-c7d6-4941-81eb-e4fa1f280fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:56.403746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9f49-c7d6-4941-81eb-e4fa1f280fa0 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:56.404241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9f49-c7d6-4941-81eb-e4fa1f280fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:56.404700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9f49-c7d6-4941-81eb-e4fa1f280fa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:56.405851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e9f49-c7d6-4941-81eb-e4fa1f280fa0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:56.416458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e9f49-c7d6-4941-81eb-e4fa1f280fa0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:56.420438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11114515-e6d3-4fa8-8bcd-297bf5b91415 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:56.479035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 418/826] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:05:56 2022] POST /identity/v3/endpoints => generated 337 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:05:56.511646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:56.512396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:56.512777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:56.513057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:56.513856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:56.514228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:56.516054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:56.542596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 409/827] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 31 12:05:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:56.545476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e9f49-c7d6-4941-81eb-e4fa1f280fa0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:56.614911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:56.712733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e9f49-c7d6-4941-81eb-e4fa1f280fa0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:56.754964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:56.755946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:56.756425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:56.756864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:56.771095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:56.773682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:56.792792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:56.793864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:56.794805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:56.794805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:56.794805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:56.801551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 419/828] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:56.803124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:56.948862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:56.970516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cc757-d33d-4825-b9a8-e3a6b4c0c742 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:57.021159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82815a6-3032-4096-84ce-f16a9a70169f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:57.022191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82815a6-3032-4096-84ce-f16a9a70169f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:57.022673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82815a6-3032-4096-84ce-f16a9a70169f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:57.023145 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82815a6-3032-4096-84ce-f16a9a70169f None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:57.024338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82815a6-3032-4096-84ce-f16a9a70169f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:57.024866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82815a6-3032-4096-84ce-f16a9a70169f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:57.027371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82815a6-3032-4096-84ce-f16a9a70169f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:57.059901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 410/829] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:57.065377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:57.220375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:57.244324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a4adb-40a4-4da3-9ab1-bff40d739814 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:57.303215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:57.303878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:57.304148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:57.304446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:57.312223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:57.313684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:57.321003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8777f61-df19-4a5f-8126-34ab3254d3f6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:57.321514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8777f61-df19-4a5f-8126-34ab3254d3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:57.321889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8777f61-df19-4a5f-8126-34ab3254d3f6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:05:57.326731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:57.327751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:57.328653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:05:57.328653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:05:57.328653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:05:57.328653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:05:57.328653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:05:57.328653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:05:57.328653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:05:57.328653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:57.331112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 411/831] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:57.332809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 420/831] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:57.333794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df823c2-51d2-4146-a79b-bee73db6b3e3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:57.334037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:57.334296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df823c2-51d2-4146-a79b-bee73db6b3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:57.334790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df823c2-51d2-4146-a79b-bee73db6b3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:57.365461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7df823c2-51d2-4146-a79b-bee73db6b3e3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:57.367501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7df823c2-51d2-4146-a79b-bee73db6b3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:57.462785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b43aa-4137-47be-893b-9a499afeb8de None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:57.490697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 412/832] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:05:57.527296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:57.528407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:57.528793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:57.529270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:57.530548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:57.531089 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:57.533777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:57.538286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9a0e-9160-4a21-8861-8731a7e45cc0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:57.538814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9a0e-9160-4a21-8861-8731a7e45cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:57.539303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9a0e-9160-4a21-8861-8731a7e45cc0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:57.542017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 413/833] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:57.549374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0df7a-b5ec-4f87-a27c-6b9828c77b33 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:57.549374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0df7a-b5ec-4f87-a27c-6b9828c77b33 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:57.549374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0df7a-b5ec-4f87-a27c-6b9828c77b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:57.563430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 421/834] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:57.568407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:57.569139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3df0df7a-b5ec-4f87-a27c-6b9828c77b33 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:57.570555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3df0df7a-b5ec-4f87-a27c-6b9828c77b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:57.586965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:57.667875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 414/835] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:05:57.727642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9b73c-c481-48a8-bda2-0a11a2b117b1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:57.790082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2da6e-e9e3-4f5b-86e8-b09d4a4e833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:57.791113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2da6e-e9e3-4f5b-86e8-b09d4a4e833f None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:57.791644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2da6e-e9e3-4f5b-86e8-b09d4a4e833f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:57.792131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2da6e-e9e3-4f5b-86e8-b09d4a4e833f None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/ce1444d8c66045389eac37f8a7309378/roles/aafb8817e1d640eb961fa58db7e05111` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:57.819700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2da6e-e9e3-4f5b-86e8-b09d4a4e833f None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=ce1444d8c66045389eac37f8a7309378, role_id=aafb8817e1d640eb961fa58db7e05111)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:57.822415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2da6e-e9e3-4f5b-86e8-b09d4a4e833f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:57.851425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 422/836] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:05:57 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/ce1444d8c66045389eac37f8a7309378/roles/aafb8817e1d640eb961fa58db7e05111 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:05:57.971214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0df7a-b5ec-4f87-a27c-6b9828c77b33 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:57.975965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:57.976808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2da6e-e9e3-4f5b-86e8-b09d4a4e833f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:58.049130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:58.112246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2da6e-e9e3-4f5b-86e8-b09d4a4e833f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:58.125551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0df7a-b5ec-4f87-a27c-6b9828c77b33 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:58.169831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:58.170875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.171397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.171883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.172768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:58.174477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:58.181474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3cc9b-17cd-4d69-befe-c185a136e012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:58.182450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3cc9b-17cd-4d69-befe-c185a136e012 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.182869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3cc9b-17cd-4d69-befe-c185a136e012 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.183328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3cc9b-17cd-4d69-befe-c185a136e012 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:58.186372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:58.187687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:05:58.188544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:58.191463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 423/837] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:58.197088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:58.203855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3cc9b-17cd-4d69-befe-c185a136e012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-24b3cc9b-17cd-4d69-befe-c185a136e012 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:05:58.208671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:58.210279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:58.211782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 415/838] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:58.216786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3cc9b-17cd-4d69-befe-c185a136e012 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:58.339366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:58.352769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d37da-0e1a-40a4-9dc4-92fb9b3b90f1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:58.382351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:58.396402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:58.397165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.397633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.397899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.398630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:58.399027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:58.400577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:58.411729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 424/839] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:58.412810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:58.437386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3cc9b-17cd-4d69-befe-c185a136e012 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:58.516701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:58.517555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.517933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.518323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.519154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:58.519919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:58.521151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:58.536103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 416/840] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:05:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:58.537175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:58.565907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf5d32-ae1f-46d0-a801-f44dcf645689 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:58.605328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343ee11-c7da-40ea-b6ee-d8b72ed6837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:58.605877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343ee11-c7da-40ea-b6ee-d8b72ed6837a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.606347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343ee11-c7da-40ea-b6ee-d8b72ed6837a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.606619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343ee11-c7da-40ea-b6ee-d8b72ed6837a None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.607285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343ee11-c7da-40ea-b6ee-d8b72ed6837a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:58.607582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343ee11-c7da-40ea-b6ee-d8b72ed6837a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:58.608966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343ee11-c7da-40ea-b6ee-d8b72ed6837a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:58.618875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 425/841] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:05:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:05:58.618875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee908-126b-41f0-b980-01458e875070 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.618875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee908-126b-41f0-b980-01458e875070 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.618875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee908-126b-41f0-b980-01458e875070 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.618875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 426/842] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:05:58.627445 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8f336-491c-4dd5-a521-421666991e92 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.627890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8f336-491c-4dd5-a521-421666991e92 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.628400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8f336-491c-4dd5-a521-421666991e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.658367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2dd8f336-491c-4dd5-a521-421666991e92 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:58.660383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2dd8f336-491c-4dd5-a521-421666991e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:58.701790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e210fe-c02a-48ce-a33c-40fa963fa5f4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:58.740702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:58.745816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:58.756806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:58.757828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.758275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.758745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.776688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 427/843] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:05:58.779949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:58.783125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:05:58.809186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:58.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:05:58.812997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:05:58.812997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:05:58.812997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:58.812997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 417/844] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:58.820692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8f336-491c-4dd5-a521-421666991e92 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:58.934272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8f336-491c-4dd5-a521-421666991e92 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:58.971693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:58.972620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:58.972939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:58.973196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:58.973857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:58.974155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:58.975580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:58.988604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 428/845] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:58.997722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:59.155157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:59.181213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4b06e-95e3-4dbf-83d0-d4486a841aba None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:59.219288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:59.242862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:59.244528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:59.245357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:59.246094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:59.262301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:59.265196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60301d6-4f78-4ee1-b212-eb8b309fdc50 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:59.265661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:05:59.289199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:59.290953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:05:59.292760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 418/846] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:05:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:05:59.301582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:59.302227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:59.302485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:59.302760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:59.303791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:59.304900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:59.308854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:05:59.314892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:05:59.315999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:05:59.316911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 429/847] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 31 12:05:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:05:59.322137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:59.387819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:59.437928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41450c-08f1-4799-923b-99a93803ba17 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:59.441404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:59.472143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481259a7-f5d5-4248-aa9f-4c48fcdeeb7d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:59.488646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:59.490050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:59.490643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:59.491544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:59.493155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:59.494006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:59.496480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:59.503949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 430/848] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:05:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:59.509228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:59.522984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:59.524455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:59.525192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:59.525747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:59.526971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:59.528095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:59.530484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:59.555324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 419/849] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 31 12:05:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:59.557856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:59.632429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5aeea-b08b-451d-bd45-b40565eb12e4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:59.666299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:59.672853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b9d86-62e7-4697-8a60-4243005f85d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:59.673702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b9d86-62e7-4697-8a60-4243005f85d3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:59.674155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b9d86-62e7-4697-8a60-4243005f85d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:59.674492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0098a4-27a8-43be-8215-2eba7e279e6d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:05:59.674752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b9d86-62e7-4697-8a60-4243005f85d3 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:59.675560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b9d86-62e7-4697-8a60-4243005f85d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:05:59.676006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b9d86-62e7-4697-8a60-4243005f85d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:59.677810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b9d86-62e7-4697-8a60-4243005f85d3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:59.691798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 431/850] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:05:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:05:59.716007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477639b5-f3fe-4e23-bf2f-bdd4246e5aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:05:59.716882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477639b5-f3fe-4e23-bf2f-bdd4246e5aea None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:59.717224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477639b5-f3fe-4e23-bf2f-bdd4246e5aea None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:59.717558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477639b5-f3fe-4e23-bf2f-bdd4246e5aea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:59.807283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477639b5-f3fe-4e23-bf2f-bdd4246e5aea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:05:59.808817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477639b5-f3fe-4e23-bf2f-bdd4246e5aea None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:05:59.899266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c55e2-d0e9-4828-a30e-f28fb9022bcf None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:59.899570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c55e2-d0e9-4828-a30e-f28fb9022bcf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:59.899849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c55e2-d0e9-4828-a30e-f28fb9022bcf None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:59.902054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 432/851] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:05:59.908286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a2cc-e611-4c88-b6ab-efeab57dded3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:05:59.908562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a2cc-e611-4c88-b6ab-efeab57dded3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:05:59.909135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a2cc-e611-4c88-b6ab-efeab57dded3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:05:59.925239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:05:59.933520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3c65a2cc-e611-4c88-b6ab-efeab57dded3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:05:59.933714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 420/852] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:05:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 12:05:59.934855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3c65a2cc-e611-4c88-b6ab-efeab57dded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:05:59.939463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477639b5-f3fe-4e23-bf2f-bdd4246e5aea None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:05:59.964329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:00.011820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 433/853] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:05:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:00.066079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477639b5-f3fe-4e23-bf2f-bdd4246e5aea None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:00.103692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:00.104335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:00.104614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:00.104887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:00.115891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:00.117384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:00.134696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:00.135803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:00.137258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:00.137258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:00.137258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:00.137258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 421/854] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:00.141908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65a2cc-e611-4c88-b6ab-efeab57dded3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:00.253535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65a2cc-e611-4c88-b6ab-efeab57dded3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:00.289249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:00.289863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:00.290140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:00.290415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:00.291063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:00.291398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:00.292780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:00.301259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:00.307282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 434/855] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:00.312283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:00.339822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:00.365932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:00.429528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277594cd-1f0c-4bde-915d-51c22d5fd4ef None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:00.430468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdebb8d-8e02-427c-93e3-de331b96b65f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:00.466563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e087f-d872-4b4d-9753-6653b917d0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:00.466776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:00.467353 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:00.467629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:00.467938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:00.468363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e087f-d872-4b4d-9753-6653b917d0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:00.468982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e087f-d872-4b4d-9753-6653b917d0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:00.469472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e087f-d872-4b4d-9753-6653b917d0c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:00.475767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:00.477274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:00.481821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e087f-d872-4b4d-9753-6653b917d0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-182e087f-d872-4b4d-9753-6653b917d0c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:00.487377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:00.488320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:06:00.489158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:00.489811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:00.490594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:00.491545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:00.491545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:00.491545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:00.491545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:00.491545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:00.491545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:00.491545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:00.491545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:00.494374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 422/857] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:00.494502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 435/857] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:00.499383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:00.501807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e087f-d872-4b4d-9753-6653b917d0c2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:00.594843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:00.611889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e087f-d872-4b4d-9753-6653b917d0c2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:00.617438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4031f3-4ba4-4e64-af11-814ef1d969d9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:00.623072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:00.649613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:00.650430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:00.650814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:00.651171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:00.652000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:00.652434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:00.653968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:00.661281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:00.662600 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:00.663066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:00.663535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:00.664399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:00.664901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:00.666735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:00.668694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 423/858] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:06:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:00.669885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:00.687348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 436/859] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:00.690308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:00.782087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b14d5a-0799-41d9-8a32-3823970b7450 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:00.820092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5d9c-6c78-4cf7-bee3-67493d14c638 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:00.827074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:00.827784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:00.828064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:00.828334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:00.842826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:00.844353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:00.863346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:00.864980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:00.867100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:00.867100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:00.867100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:00.869389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 424/860] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:00.870289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:00.878463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:00.880035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:00.880803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:00.881548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:00.883208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=981e9da7f2774c90a667fab35ecab64b) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:00.884096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:00.887178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:00.907401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 437/861] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:00 2022] GET /identity/v3/role_assignments?user.id=981e9da7f2774c90a667fab35ecab64b&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:00.911713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:00.991135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a86ab5-d60c-4272-83a0-5306fdb9acc4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:01.031210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:01.031866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:01.032144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:01.032416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:01.033069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:01.033372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:01.034784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:01.049261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 425/862] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:01.056014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:01.071287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:01.192059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95baadb5-c4b3-41c2-9021-c6426211f1c5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:01.231417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:01.232101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:01.232383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:01.232648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:01.241599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:01.243098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:06:01.256444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:01.259161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:01.261130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 438/863] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:06:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:01.267258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:01.356252 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:01.376851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d9bd8-d35a-4c7a-9a5e-e27a18b9aa09 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:01.412508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225bf8f-8288-4d52-8e8d-2d18c63d9941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:01.413122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225bf8f-8288-4d52-8e8d-2d18c63d9941 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:01.413404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225bf8f-8288-4d52-8e8d-2d18c63d9941 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:01.413731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225bf8f-8288-4d52-8e8d-2d18c63d9941 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:01.414474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225bf8f-8288-4d52-8e8d-2d18c63d9941 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:01.414777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225bf8f-8288-4d52-8e8d-2d18c63d9941 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:01.416353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225bf8f-8288-4d52-8e8d-2d18c63d9941 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:01.435920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 426/864] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 31 12:06:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:01.442407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:01.488449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f499-c67c-45aa-91f6-ad15969633d5 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:01.488750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f499-c67c-45aa-91f6-ad15969633d5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:01.489013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f499-c67c-45aa-91f6-ad15969633d5 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:01.490627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 427/865] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:01.495842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87695c-b2ed-4c94-a04b-1a57741df7ad None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:01.496127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87695c-b2ed-4c94-a04b-1a57741df7ad None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:01.496503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87695c-b2ed-4c94-a04b-1a57741df7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:01.517112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5f87695c-b2ed-4c94-a04b-1a57741df7ad None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:01.518668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5f87695c-b2ed-4c94-a04b-1a57741df7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:01.552032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:01.591690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be883b-258d-4fce-bfae-cebdd5d4920b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:01.611405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 428/866] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:01.637740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a1946-d2f7-47a0-826e-9af4fbedaa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:01.638810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a1946-d2f7-47a0-826e-9af4fbedaa5e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:01.639290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a1946-d2f7-47a0-826e-9af4fbedaa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:01.639764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a1946-d2f7-47a0-826e-9af4fbedaa5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:01.686634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399883-141b-4e5f-adf1-36860b53adef None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:01.687333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399883-141b-4e5f-adf1-36860b53adef None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:01.687808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399883-141b-4e5f-adf1-36860b53adef None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:01.689655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 429/867] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:01.695100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132138e-a079-4657-bc04-c0f9a91c3086 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:01.695353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132138e-a079-4657-bc04-c0f9a91c3086 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:01.695613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132138e-a079-4657-bc04-c0f9a91c3086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:01.713944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c132138e-a079-4657-bc04-c0f9a91c3086 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:01.715306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c132138e-a079-4657-bc04-c0f9a91c3086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:01.727637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a1946-d2f7-47a0-826e-9af4fbedaa5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:01.728952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a1946-d2f7-47a0-826e-9af4fbedaa5e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:01.800388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:01.803739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:01.812190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 430/868] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:01.856761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 439/869] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 12:06:01.862773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132138e-a079-4657-bc04-c0f9a91c3086 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:01.899045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a1946-d2f7-47a0-826e-9af4fbedaa5e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:01.990703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132138e-a079-4657-bc04-c0f9a91c3086 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:02.006965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a1946-d2f7-47a0-826e-9af4fbedaa5e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:02.038316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:02.039807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:02.040429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:02.040953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:02.045181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c940108-068b-4963-bb70-9f5c1d00637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:02.045944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c940108-068b-4963-bb70-9f5c1d00637c None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:02.046316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c940108-068b-4963-bb70-9f5c1d00637c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:02.046683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c940108-068b-4963-bb70-9f5c1d00637c None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:02.047531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c940108-068b-4963-bb70-9f5c1d00637c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:02.049049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c940108-068b-4963-bb70-9f5c1d00637c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:02.053892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:02.055868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:02.071951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:02.072992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:02.073924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:02.073924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:02.073924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:02.075025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 440/871] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:06:01 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:06:02.077801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 431/871] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:02.078851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c940108-068b-4963-bb70-9f5c1d00637c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:02.096449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:02.114097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:02.171944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:02.234964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a152f-2120-47dd-8cf5-352759ee4322 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:02.239166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c940108-068b-4963-bb70-9f5c1d00637c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:02.282594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e7a63-90fc-443d-b2b4-b2b6df43bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:02.283573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e7a63-90fc-443d-b2b4-b2b6df43bbd8 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:02.284062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e7a63-90fc-443d-b2b4-b2b6df43bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:02.284592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e7a63-90fc-443d-b2b4-b2b6df43bbd8 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:02.285486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e7a63-90fc-443d-b2b4-b2b6df43bbd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:02.288116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e7a63-90fc-443d-b2b4-b2b6df43bbd8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:02.288514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:02.289138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:02.289416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:02.289688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:02.290348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:02.290650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:02.292133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:02.307021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 441/872] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:02.312819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 432/873] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:06:02 2022] POST /identity/v3/services => generated 269 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:06:02.312995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:02.419734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:02.443351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2166d54-cd0a-49a1-9dd2-0992d9d709b2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:02.488639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:02.489695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:02.490195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:02.490640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:02.498670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:02.500455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:02.513055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:02.514022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:02.514933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:02.514933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:02.514933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:02.514933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:02.514933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:02.514933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:02.514933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:02.514933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:02.517040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 442/874] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:02.522619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e7a63-90fc-443d-b2b4-b2b6df43bbd8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:02.642764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e7a63-90fc-443d-b2b4-b2b6df43bbd8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:02.668060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:02.708399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffd39f-245a-4e77-85f3-409553cf7ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:02.709427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffd39f-245a-4e77-85f3-409553cf7ebe None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:02.709909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffd39f-245a-4e77-85f3-409553cf7ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:02.710387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffd39f-245a-4e77-85f3-409553cf7ebe None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:02.711686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffd39f-245a-4e77-85f3-409553cf7ebe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:02.712216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffd39f-245a-4e77-85f3-409553cf7ebe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:02.714905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffd39f-245a-4e77-85f3-409553cf7ebe None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:02.738062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 433/875] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:02.743593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:02.874448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd8d30-8c59-481f-8063-10027a9a8061 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:02.912754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf71f19-7afb-4369-871a-a1f91d5dc2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:02.913372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71f19-7afb-4369-871a-a1f91d5dc2ae None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:02.913649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71f19-7afb-4369-871a-a1f91d5dc2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:02.913915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71f19-7afb-4369-871a-a1f91d5dc2ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:02.914646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf71f19-7afb-4369-871a-a1f91d5dc2ae None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=ce1444d8c66045389eac37f8a7309378) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:02.914954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf71f19-7afb-4369-871a-a1f91d5dc2ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:02.916435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf71f19-7afb-4369-871a-a1f91d5dc2ae None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:02.923979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 443/876] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:02 2022] GET /identity/v3/role_assignments?user.id=ce1444d8c66045389eac37f8a7309378&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:03.648837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a948fa-69f8-47c6-b8c1-8487d274da10 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:03.649561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a948fa-69f8-47c6-b8c1-8487d274da10 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:03.649561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a948fa-69f8-47c6-b8c1-8487d274da10 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:03.650936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 434/877] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:03.656936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa23092-df77-4496-8302-77d7c5a6b413 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:03.657222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa23092-df77-4496-8302-77d7c5a6b413 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:03.657494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa23092-df77-4496-8302-77d7c5a6b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:03.663838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:03.679037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-efa23092-df77-4496-8302-77d7c5a6b413 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:03.680616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-efa23092-df77-4496-8302-77d7c5a6b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:03.753538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 444/878] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:04.027028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a948fa-69f8-47c6-b8c1-8487d274da10 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:04.148697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a948fa-69f8-47c6-b8c1-8487d274da10 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:04.187174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:04.187889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:04.188170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:04.188439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:04.189035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:04.190566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:04.191987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6cc9-dbbe-492a-ad09-098d62926aba None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:04.192262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6cc9-dbbe-492a-ad09-098d62926aba None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:04.192768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6cc9-dbbe-492a-ad09-098d62926aba None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:04.193872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 445/879] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:04.200076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81e454-2e13-4ff1-bfc2-54b3cc4bea3a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:04.200552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:04.201528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:04.202361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 435/880] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:04.204169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81e454-2e13-4ff1-bfc2-54b3cc4bea3a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:04.204515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81e454-2e13-4ff1-bfc2-54b3cc4bea3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:04.213136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:04.228769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6f81e454-2e13-4ff1-bfc2-54b3cc4bea3a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:04.230537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6f81e454-2e13-4ff1-bfc2-54b3cc4bea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:04.251255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:04.323615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 446/881] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:04.336957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab8349-b766-4bda-b415-7d44505dde9d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:04.376586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:04.377208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:04.377484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:04.377784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:04.378419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:04.378723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:04.380194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:04.385819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 436/882] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:04.392120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81e454-2e13-4ff1-bfc2-54b3cc4bea3a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:04.461809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:04.503798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81e454-2e13-4ff1-bfc2-54b3cc4bea3a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:04.544381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:04.544991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:04.545262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:04.545520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:04.546167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:04.546463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:04.547930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:04.553243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 447/883] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:04.605210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:04.721321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:04.726525 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13113b92-1112-44f4-af9e-4aa23bf129d6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:04.768583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:04.769726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:04.770217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:04.770688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:04.771524 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:04.773463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:04.784170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:04.785224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:06:04.786117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:04.789085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 437/884] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:06:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:04.796357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:04.971816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506693d9-5e05-4c63-8d6d-2c446a4dce7a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:05.008127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:05.017986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf938e-099b-4d0e-a45c-76111b387404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:05.018595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf938e-099b-4d0e-a45c-76111b387404 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.018863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf938e-099b-4d0e-a45c-76111b387404 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.019126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf938e-099b-4d0e-a45c-76111b387404 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.019800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf938e-099b-4d0e-a45c-76111b387404 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:05.020151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf938e-099b-4d0e-a45c-76111b387404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:05.021756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf938e-099b-4d0e-a45c-76111b387404 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:05.026984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 448/885] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:06:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:05.036473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:05.059638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31505b2-387d-4dc4-ac82-97cde434df6e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.060356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31505b2-387d-4dc4-ac82-97cde434df6e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.060634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31505b2-387d-4dc4-ac82-97cde434df6e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.062551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 449/886] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:05.067869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06805eac-26da-4bfc-b82e-ba92f901b67d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.068221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06805eac-26da-4bfc-b82e-ba92f901b67d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.068559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06805eac-26da-4bfc-b82e-ba92f901b67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.091349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-06805eac-26da-4bfc-b82e-ba92f901b67d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:05.092919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-06805eac-26da-4bfc-b82e-ba92f901b67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:05.176026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174d32-7271-46d0-aa9d-e0c2e1291f34 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:05.205952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 450/887] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:05.215961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86d7ea-150c-44e2-8f31-9febe4049f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:05.216883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d7ea-150c-44e2-8f31-9febe4049f5f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.217335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d7ea-150c-44e2-8f31-9febe4049f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.217764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d7ea-150c-44e2-8f31-9febe4049f5f None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.218552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86d7ea-150c-44e2-8f31-9febe4049f5f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:05.219022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86d7ea-150c-44e2-8f31-9febe4049f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:05.220586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86d7ea-150c-44e2-8f31-9febe4049f5f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:05.226948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 438/888] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:06:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:05.233250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06805eac-26da-4bfc-b82e-ba92f901b67d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:05.328957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00615a1-b1d0-48dd-80fb-798bff5557d0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.329230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00615a1-b1d0-48dd-80fb-798bff5557d0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.329501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00615a1-b1d0-48dd-80fb-798bff5557d0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.331081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 439/889] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:05.336115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6e7fb-649a-44f1-90f1-d8091672f1f8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.336396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6e7fb-649a-44f1-90f1-d8091672f1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.336664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6e7fb-649a-44f1-90f1-d8091672f1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.357569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-adc6e7fb-649a-44f1-90f1-d8091672f1f8 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:05.358866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-adc6e7fb-649a-44f1-90f1-d8091672f1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:05.383970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:05.407440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06805eac-26da-4bfc-b82e-ba92f901b67d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:05.440209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:05.455317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:05.456075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.456439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.456797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.457596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] RBAC: Adding query filter params (interface=public, service_id=de731e0240854f6fafcaa67ac9602081, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:05.458013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:05.459567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:05.464593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 440/890] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:05.468735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 451/891] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 31 12:06:05 2022] GET /identity/v3/endpoints?service_id=de731e0240854f6fafcaa67ac9602081&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:05.478578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6e7fb-649a-44f1-90f1-d8091672f1f8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:05.571774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:05.601175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6e7fb-649a-44f1-90f1-d8091672f1f8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:05.636241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1676b-70e8-4fe4-96f9-12745edda69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:05.636942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1676b-70e8-4fe4-96f9-12745edda69d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.637318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1676b-70e8-4fe4-96f9-12745edda69d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.637590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1676b-70e8-4fe4-96f9-12745edda69d None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.638192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1676b-70e8-4fe4-96f9-12745edda69d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:05.639608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1676b-70e8-4fe4-96f9-12745edda69d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:05.645728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 441/892] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:06:05 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:06:05.751143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1676b-70e8-4fe4-96f9-12745edda69d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:05.769460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8ec72-58a9-4826-94f8-1d951f7d4466 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:05.795858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:05.819319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49f557-a99c-4415-b384-8ba042358207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:05.820191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f557-a99c-4415-b384-8ba042358207 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.820504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f557-a99c-4415-b384-8ba042358207 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.820810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f557-a99c-4415-b384-8ba042358207 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.845941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49f557-a99c-4415-b384-8ba042358207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8c49f557-a99c-4415-b384-8ba042358207 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:05.851830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:05.853658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:05.856131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 452/893] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 31 12:06:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:05.861792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49f557-a99c-4415-b384-8ba042358207 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:05.931200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1676b-70e8-4fe4-96f9-12745edda69d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:05.980558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:05.981868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:05.982468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:05.982958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:05.983834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:05.985512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:05.986571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:05.999423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:06.000451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:06:06.001434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:06.004613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 442/894] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 31 12:06:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:06.018730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:06.085567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49f557-a99c-4415-b384-8ba042358207 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:06.123873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:06.124972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.125459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.125923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.126780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:06.127304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:06.128890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:06.143196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 453/895] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 31 12:06:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:06.144957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:06.152957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:06.171090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d19b-3847-4d75-ad05-baf2e582d33a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:06.212601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:06.213182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.213502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.213790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.214446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:06.214810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:06.216519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:06.227555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 443/896] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:06.228594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:06.289362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52cb93-1259-470b-a22f-02dc448f604d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:06.299718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:06.337952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:06.339203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.340114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.340812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.342919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be01f6-cc6d-4277-bf65-2957f4c5095c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:06.356804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:06.359137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:06.380526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:06.381546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:06.382441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:06.382441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:06.382441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:06.383757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:06.388731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.388731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.388731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.388731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:06.388731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:06.388731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:06.389864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 454/897] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:06.392038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:06.398825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 444/898] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:06.399974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:06.417729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:06.515512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051680af-749b-4352-9140-4febcffcc3fb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:06.539356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb41b4f-2a7b-42fe-932b-78a671fe1f59 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:06.554496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:06.555696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.556583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.556999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.558052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:06.558713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:06.561401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:06.582681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 455/899] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:06.583580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:06.584183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.584394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.584667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.585383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] RBAC: Adding query filter params (interface=public, service_id=c69eef5e31cc4112b8981603061e62c5, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:06.585688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:06.587110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:06.589659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:06.599799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 445/900] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 31 12:06:06 2022] GET /identity/v3/endpoints?service_id=c69eef5e31cc4112b8981603061e62c5&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:06.600748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:06.627284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:06.693058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:06.719545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac28ad-8314-4290-a0fe-dfdce2f43d6e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:06.751185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9b626-0934-49a2-b7e6-9b6cf86032ca None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:06.759873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b934d3a-3d84-430c-9357-d11aad336ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:06.760555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b934d3a-3d84-430c-9357-d11aad336ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.760783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b934d3a-3d84-430c-9357-d11aad336ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.761055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b934d3a-3d84-430c-9357-d11aad336ff4 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.761716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b934d3a-3d84-430c-9357-d11aad336ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:06.763716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b934d3a-3d84-430c-9357-d11aad336ff4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:06.770952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 446/901] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:06:06 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:06:06.773082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cdd1-2fe2-4189-997b-ae55d21e8c6a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.773570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cdd1-2fe2-4189-997b-ae55d21e8c6a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.774022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cdd1-2fe2-4189-997b-ae55d21e8c6a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.776435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 447/902] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:06.782368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ca3b7-136e-48f8-ab7e-f83806a179e1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.783648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ca3b7-136e-48f8-ab7e-f83806a179e1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.783648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ca3b7-136e-48f8-ab7e-f83806a179e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.822477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:06.824866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7c0ca3b7-136e-48f8-ab7e-f83806a179e1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:06.825516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:06.826244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:06.826918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7c0ca3b7-136e-48f8-ab7e-f83806a179e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:06.827300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:06.851578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:06.854901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:06:06.879765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:06.881684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:06:06.883624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:06.888419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 456/903] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:06:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:06.898104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:06.943364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 448/904] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:07.045273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:07.078554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6455194-a8e5-424e-b889-ca1d5d0a6412 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:07.121793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:07.122440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:07.122723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:07.122988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:07.128847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:07.129617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:07.132543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:07.167032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 457/905] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 31 12:06:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:07.173302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ca3b7-136e-48f8-ab7e-f83806a179e1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:07.240257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:07.248869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:07.292671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ca3b7-136e-48f8-ab7e-f83806a179e1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:07.341985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9147c9-75ea-4e27-84ff-3c78a49adc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:07.342531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9147c9-75ea-4e27-84ff-3c78a49adc52 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:07.342809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9147c9-75ea-4e27-84ff-3c78a49adc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:07.343076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9147c9-75ea-4e27-84ff-3c78a49adc52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:07.371123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7d6a0-b2b6-4d6b-8fa4-f5ae9168530d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:07.417601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea29be-ff9a-4225-87a1-d5a6796eeee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:07.418298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea29be-ff9a-4225-87a1-d5a6796eeee0 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:07.418598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea29be-ff9a-4225-87a1-d5a6796eeee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:07.418849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea29be-ff9a-4225-87a1-d5a6796eeee0 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:07.419687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea29be-ff9a-4225-87a1-d5a6796eeee0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:07.421138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea29be-ff9a-4225-87a1-d5a6796eeee0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:07.428077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:07.451271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 458/906] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:06:07 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:07.451482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9147c9-75ea-4e27-84ff-3c78a49adc52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:07.451482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9147c9-75ea-4e27-84ff-3c78a49adc52 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:07.555540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:07.613063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 449/907] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:07 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:06:07.620813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea29be-ff9a-4225-87a1-d5a6796eeee0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:07.653475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d154c16-bfa3-441a-a954-df64ef490aff None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:07.653967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d154c16-bfa3-441a-a954-df64ef490aff None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:07.654408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d154c16-bfa3-441a-a954-df64ef490aff None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:07.656863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 450/908] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:07.663793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890bbd-fa30-47aa-81d6-febaf3107375 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:07.664315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890bbd-fa30-47aa-81d6-febaf3107375 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:07.664760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890bbd-fa30-47aa-81d6-febaf3107375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:07.693235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3d890bbd-fa30-47aa-81d6-febaf3107375 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:07.695021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3d890bbd-fa30-47aa-81d6-febaf3107375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:07.768929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea29be-ff9a-4225-87a1-d5a6796eeee0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:07.789683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 451/909] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:07.835122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:07.836332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:07.836807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:07.837269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:07.857346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:07.859799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:07.884973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:07.885955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:07.886997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:07.886997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:07.886997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:07.887377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 459/910] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:07.899615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d890bbd-fa30-47aa-81d6-febaf3107375 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:08.047703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d890bbd-fa30-47aa-81d6-febaf3107375 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:08.049694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:08.064817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:08.095657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:08.096436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:08.096810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:08.097515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:08.098279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:08.098677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:08.100214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:08.118020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 452/911] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:08.121375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:08.133113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:08.199003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b2d2c-2421-48b1-96bc-7bc73974a432 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:08.235090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f67f-af6a-40a2-a625-6e1585775d7e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:08.237836 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:08.238796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:08.239316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:08.239803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:08.240601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:08.242173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:08.252430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:08.253553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 12:06:08.254420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:08.262102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 460/912] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:06:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:08.263340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:08.272691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:08.273329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:08.273602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:08.273868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:08.281098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:08.282557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:08.294471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:08.295471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:08.296412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:08.296412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:08.296412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:08.296412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:08.296412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:08.296412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:08.296412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:08.296412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:08.300539 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 453/913] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:08.301872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:08.329234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:08.378681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210244d-6098-4e22-9a8a-20928fc2936d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:08.390191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:08.415744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f9a8-c1fc-46ab-9e32-452bb8018126 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:08.420257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:08.421201 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:08.421679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:08.422172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:08.423008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:08.423542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:08.425146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:08.430783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 461/914] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:06:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:08.442968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:08.453688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:08.454208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:08.454511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:08.454789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:08.460997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:08.461278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:08.462762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:08.483791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 454/915] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:06:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:08.486298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:08.557975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ae31-c839-4766-81f2-9194b91a842b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:08.585241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:08.596262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:08.596967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:08.597647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:08.598011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:08.598749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:08.599149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:08.600776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:08.609892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4568f7-037b-486d-9c0b-7062cbf7d481 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:08.611897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 462/916] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 31 12:06:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:08.612868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:08.649790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de5108-faf8-4de1-908b-871dd3c82c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:08.650767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de5108-faf8-4de1-908b-871dd3c82c27 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:08.650895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:08.651468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de5108-faf8-4de1-908b-871dd3c82c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:08.651956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de5108-faf8-4de1-908b-871dd3c82c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:08.653545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de5108-faf8-4de1-908b-871dd3c82c27 None admin] RBAC: Adding query filter params (role.id=0b69eefec9d249b39b3ef87c6de99c50, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=ce1444d8c66045389eac37f8a7309378) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:08.654259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de5108-faf8-4de1-908b-871dd3c82c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:08.655965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de5108-faf8-4de1-908b-871dd3c82c27 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:08.667471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 455/917] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:08 2022] GET /identity/v3/role_assignments?user.id=ce1444d8c66045389eac37f8a7309378&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=0b69eefec9d249b39b3ef87c6de99c50 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:08.730975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7168-8c1f-42db-9889-04b08de00e18 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:08.772988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66971b7d-d82b-446c-80a6-df02aeedd8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:08.773695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66971b7d-d82b-446c-80a6-df02aeedd8fc None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:08.773984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66971b7d-d82b-446c-80a6-df02aeedd8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:08.774258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66971b7d-d82b-446c-80a6-df02aeedd8fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:08.774951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66971b7d-d82b-446c-80a6-df02aeedd8fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:08.776393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66971b7d-d82b-446c-80a6-df02aeedd8fc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:08.802351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 463/918] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:06:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:09.024208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f69a3-6b4b-4ba6-a457-57c0dabbe30a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:09.025286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f69a3-6b4b-4ba6-a457-57c0dabbe30a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:09.025775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f69a3-6b4b-4ba6-a457-57c0dabbe30a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:09.028056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 456/919] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:09.034065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca1fb4-640f-4626-8e35-9a9b95407f2b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:09.034637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca1fb4-640f-4626-8e35-9a9b95407f2b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:09.034637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca1fb4-640f-4626-8e35-9a9b95407f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:09.054834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-40ca1fb4-640f-4626-8e35-9a9b95407f2b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:09.056123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-40ca1fb4-640f-4626-8e35-9a9b95407f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:09.140488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 464/920] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:09.410834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f69a3-6b4b-4ba6-a457-57c0dabbe30a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:09.521560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f69a3-6b4b-4ba6-a457-57c0dabbe30a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:09.559607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:09.560502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:09.560787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:09.561062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:09.561656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:09.563061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:09.573033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:09.574360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:09.575758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 457/921] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 31 12:06:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:09.580652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca1fb4-640f-4626-8e35-9a9b95407f2b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:09.600909 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:09.689320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca1fb4-640f-4626-8e35-9a9b95407f2b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:09.728193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:09.728810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:09.729088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:09.729360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:09.730000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:09.730302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:09.731834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:09.737409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 465/922] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:09.742848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:09.756916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:09.855027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5d56e-935c-4e51-b91f-484fef8ed04e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:09.918852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520592df-413a-422e-b9d9-c27e56859757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:09.919890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520592df-413a-422e-b9d9-c27e56859757 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:09.920344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520592df-413a-422e-b9d9-c27e56859757 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:09.920769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520592df-413a-422e-b9d9-c27e56859757 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:09.921648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520592df-413a-422e-b9d9-c27e56859757 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:09.922105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520592df-413a-422e-b9d9-c27e56859757 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:09.923690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520592df-413a-422e-b9d9-c27e56859757 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:09.929787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 458/923] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:09.936177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:09.961783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b9201-f3aa-45c0-a9f2-3477887a2bca None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:09.962112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b9201-f3aa-45c0-a9f2-3477887a2bca None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:09.962198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b9201-f3aa-45c0-a9f2-3477887a2bca None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:09.966657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 459/924] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:09.973207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d07f-6977-491b-9016-9c3024e6122c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:09.973622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d07f-6977-491b-9016-9c3024e6122c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:09.973966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d07f-6977-491b-9016-9c3024e6122c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:09.999190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b224d07f-6977-491b-9016-9c3024e6122c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:10.000941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b224d07f-6977-491b-9016-9c3024e6122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:10.056742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:10.082735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428c4a0-31ee-46b4-b308-af16444caf0e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:10.099447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 460/925] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:10.132972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69c481-b215-4feb-ac60-60817f536766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:10.134013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69c481-b215-4feb-ac60-60817f536766 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:10.134500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69c481-b215-4feb-ac60-60817f536766 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:10.134976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69c481-b215-4feb-ac60-60817f536766 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:10.135921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69c481-b215-4feb-ac60-60817f536766 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:10.138423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69c481-b215-4feb-ac60-60817f536766 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:10.162178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 466/926] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:06:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:10.398127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d07f-6977-491b-9016-9c3024e6122c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:10.401775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:10.513432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d07f-6977-491b-9016-9c3024e6122c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:10.555816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:10.556526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:10.556804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:10.557067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:10.557645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:10.559108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:10.568159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:10.569197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:10.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 461/927] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:10.578239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69c481-b215-4feb-ac60-60817f536766 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:10.696447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69c481-b215-4feb-ac60-60817f536766 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:10.738238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:10.739262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:10.739741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:10.740220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:10.741070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:10.741583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:10.743226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:10.749974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 467/928] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:10.756004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:10.826002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:10.868844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0725f06-a35f-4bd4-a0e5-efd45dba3657 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:10.921813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e49ad1-1b9f-4acb-ae48-c5d85ecfd183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:10.922448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49ad1-1b9f-4acb-ae48-c5d85ecfd183 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:10.922719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49ad1-1b9f-4acb-ae48-c5d85ecfd183 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:10.922984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49ad1-1b9f-4acb-ae48-c5d85ecfd183 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:10.923662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e49ad1-1b9f-4acb-ae48-c5d85ecfd183 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:10.923961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e49ad1-1b9f-4acb-ae48-c5d85ecfd183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:10.925402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e49ad1-1b9f-4acb-ae48-c5d85ecfd183 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:10.932078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 462/929] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:10.942637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:10.946818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:10.968399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9428-1523-4880-a574-7c538b75114e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:10.968742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9428-1523-4880-a574-7c538b75114e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:10.968934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9428-1523-4880-a574-7c538b75114e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:10.970613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 463/930] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:10.977132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7901ea-6f0b-4f49-ad45-192044a6a8db None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:10.977132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7901ea-6f0b-4f49-ad45-192044a6a8db None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:10.977386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7901ea-6f0b-4f49-ad45-192044a6a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:10.997057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8a7901ea-6f0b-4f49-ad45-192044a6a8db None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:10.998406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a7901ea-6f0b-4f49-ad45-192044a6a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:11.079392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 464/931] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:11.079938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2adad-ee0a-4ce7-b1f9-e300caae17ee None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:11.121821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:11.122961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:11.123514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:11.123996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:11.124922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] RBAC: Adding query filter params (interface=public, service_id=f60a1a1ceb70478fb835a3311cad73ae, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:11.125479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:11.127192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:11.134194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 468/932] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 31 12:06:10 2022] GET /identity/v3/endpoints?service_id=f60a1a1ceb70478fb835a3311cad73ae&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:11.140883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7901ea-6f0b-4f49-ad45-192044a6a8db None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:11.208232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:11.257713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7901ea-6f0b-4f49-ad45-192044a6a8db None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:11.299697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af6aed-a3b0-4616-a1d2-35175d12dc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:11.300316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6aed-a3b0-4616-a1d2-35175d12dc3b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:11.300590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6aed-a3b0-4616-a1d2-35175d12dc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:11.300856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6aed-a3b0-4616-a1d2-35175d12dc3b None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:11.301499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af6aed-a3b0-4616-a1d2-35175d12dc3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:11.302880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af6aed-a3b0-4616-a1d2-35175d12dc3b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:11.310215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 465/933] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:06:11 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:06:11.377986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:11.448854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75274d-f89e-4bc2-9d8f-a568f4292f93 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:11.449790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75274d-f89e-4bc2-9d8f-a568f4292f93 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:11.449790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75274d-f89e-4bc2-9d8f-a568f4292f93 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:11.451167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 466/934] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:11.456749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d26520-4bd7-4ae1-be1c-e6bfa80a27d3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:11.457074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d26520-4bd7-4ae1-be1c-e6bfa80a27d3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:11.457282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d26520-4bd7-4ae1-be1c-e6bfa80a27d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:11.460752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:11.480658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b4d26520-4bd7-4ae1-be1c-e6bfa80a27d3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:11.482023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b4d26520-4bd7-4ae1-be1c-e6bfa80a27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:11.505463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0e602-453d-4222-9ea9-d1644d413a59 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:11.545653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:11.546275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:11.546550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:11.546817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:11.547567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:11.549106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:11.551863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING keystone.server.flask.common [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 12:06:11.574756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.sql.core [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 12:06:11.574756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 12:06:11.574756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [parameters: {'id': '51a421ac035649cb8f2183d5264da416', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 12:06:11.574756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81488) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 12:06:11.591012 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 467/935] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 51a421ac035649cb8f2183d5264da416.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 51a421ac035649cb8f2183d5264da416. Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 12:06:11.592090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 12:06:11.592934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application yield resource Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise self.value Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application session.commit() Aug 31 12:06:11.593921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.flush() Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application compat.raise_( Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 12:06:11.595047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 12:06:11.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 12:06:11.596878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [parameters: {'id': '51a421ac035649cb8f2183d5264da416', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:11.598054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:11.598846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 51a421ac035649cb8f2183d5264da416. Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:11.599970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 469/936] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Aug 31 12:06:11 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:11.602247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d26520-4bd7-4ae1-be1c-e6bfa80a27d3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:11.658074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:11.708954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d26520-4bd7-4ae1-be1c-e6bfa80a27d3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:11.745551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:11.746481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:11.746921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:11.747374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:11.754956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:11.756635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:11.768856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:11.770039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:11.771205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:11.771205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:11.771205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:11.771205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:11.771205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:11.771205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:11.771205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:11.771205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:11.773379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 468/937] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:11.779816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:11.877400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:11.957908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f808f-3740-48d2-ab7a-6cb7b7c4b351 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:11.985959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:12.004997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:12.006179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.006681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.007151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.008111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:12.008630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:12.010676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:12.036617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 470/938] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:06:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:12.049971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5dba2-eada-44dd-958a-4c13d963cc87 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:12.069126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:12.101693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b264f5-19d7-4343-a3e8-0d65b4061a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:12.102871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b264f5-19d7-4343-a3e8-0d65b4061a60 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.103435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b264f5-19d7-4343-a3e8-0d65b4061a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.103933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b264f5-19d7-4343-a3e8-0d65b4061a60 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.118753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b264f5-19d7-4343-a3e8-0d65b4061a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-67b264f5-19d7-4343-a3e8-0d65b4061a60 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:12.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:12.126695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:12.127572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:12.130521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 469/939] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 31 12:06:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:12.137953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:12.209057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2f1e-9078-4cdb-b025-c8549da36f46 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.209590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2f1e-9078-4cdb-b025-c8549da36f46 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.209821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2f1e-9078-4cdb-b025-c8549da36f46 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.211372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 470/940] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:12.217588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9571a81-1163-4e80-a47d-e52e13c53c1d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.218230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9571a81-1163-4e80-a47d-e52e13c53c1d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.218732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9571a81-1163-4e80-a47d-e52e13c53c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.243712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c9571a81-1163-4e80-a47d-e52e13c53c1d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:12.245229 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c9571a81-1163-4e80-a47d-e52e13c53c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:12.276405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3a6a4-9cd3-4b6f-9549-7a3d089e4805 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:12.336459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:12.337395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.337814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.338226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.339310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:12.339871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:12.342101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:12.353946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 471/941] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:12.355506 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 471/942] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 31 12:06:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:12.363962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9571a81-1163-4e80-a47d-e52e13c53c1d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:12.510799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9571a81-1163-4e80-a47d-e52e13c53c1d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:12.513265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:12.552317 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:12.553214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.553674 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.554143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.566936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:12.568801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:12.585999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:12.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:12.588441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:12.588441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:12.588441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:12.590374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 472/943] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:12.595939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:12.656894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:12.666806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:12.717791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b461b-1f04-4b80-af55-4f16c47504e0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:12.756005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:12.756704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.757046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.757316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.757966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:12.758307 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:12.759725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:12.772994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 472/944] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:12.779057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:12.790827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1d533-7eb6-4503-905c-af3cc2e0f287 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:12.830462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:12.831489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.831983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.832536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.833461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:12.835088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:12.845486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:12.846561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:12.847761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:12.855593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 473/945] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:12.857478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:12.878556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:12.893503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863215e-20d2-45e8-adab-3fdd60bec8cc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:12.898740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:12.943443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:12.944884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:12.945395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:12.945882 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:12.957368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:12.959525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:06:12.974404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:12.975917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:06:12.977005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:12.979583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 473/946] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:06:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:12.986288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:13.000108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbb891-ed10-4a32-a04a-f73b1584f4a8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:13.040907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:13.042111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:13.042592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:13.043517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:13.044697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:13.045209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:13.046919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:13.060254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 474/947] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:06:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:13.060863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:13.121177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a50bf-32e2-44aa-a86b-3cf6d6220170 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:13.164489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:13.169739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:13.171363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:13.171799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:13.172447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:13.173584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:13.174648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:13.176722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:13.191993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3f51-6e18-44f5-ab37-65d85c86088f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:13.211085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 474/948] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 31 12:06:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:13.217868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:13.242934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976050f-a85b-4888-a67c-610617dde9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:13.244425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976050f-a85b-4888-a67c-610617dde9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:13.244905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976050f-a85b-4888-a67c-610617dde9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:13.245367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976050f-a85b-4888-a67c-610617dde9a1 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:13.246349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976050f-a85b-4888-a67c-610617dde9a1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:13.247045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976050f-a85b-4888-a67c-610617dde9a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:13.249373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976050f-a85b-4888-a67c-610617dde9a1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:13.259555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 475/949] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:06:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:13.267766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:13.388423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007c4c2-8088-439b-86fe-b17a8c8e0566 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:13.431937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:13.432200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ced1-88c7-4f9f-b9a6-51af012c8eb3 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:13.432200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ced1-88c7-4f9f-b9a6-51af012c8eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:13.432500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ced1-88c7-4f9f-b9a6-51af012c8eb3 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:13.433773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:13.434332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:13.434635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 476/950] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:13.435096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:13.440594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e624-45db-4a18-8cce-ef777aa4d985 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:13.440944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e624-45db-4a18-8cce-ef777aa4d985 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:13.441122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e624-45db-4a18-8cce-ef777aa4d985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:13.450753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:13.452715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:13.466166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6574e624-45db-4a18-8cce-ef777aa4d985 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:13.468014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6574e624-45db-4a18-8cce-ef777aa4d985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:13.473152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:13.474384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:13.475554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:13.475554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:13.475554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:13.477121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 475/951] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:13.485604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:13.589948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:13.610605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 477/952] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:13.637623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562b86d-c86e-4748-9086-635749c1b4f0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:13.691203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:13.692752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:13.693532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:13.694284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:13.695801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:13.696669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:13.699569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:13.726356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 476/953] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:13.732485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574e624-45db-4a18-8cce-ef777aa4d985 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:13.837198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:13.876223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574e624-45db-4a18-8cce-ef777aa4d985 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:13.943789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:13.945318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:13.946073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:13.946836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:13.960516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:13.963614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:13.985778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:13.987636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:13.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:13.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:13.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:13.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:13.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:13.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:13.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:13.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:13.992909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 478/954] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:14.001042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:14.150887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:14.201677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c8905-0d0a-4697-9384-1ace3aec10c5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:14.257389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:14.258405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:14.259019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:14.259481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:14.260348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:14.261210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:14.263059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:14.265096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:14.279839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 477/955] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:06:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:14.289357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:14.349322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da536c07-8a5e-4c60-8ae2-63af080ea01f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:14.426133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:14.428475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:14.429186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:14.430084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:14.432163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:14.436249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:14.437387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d1e54-fd4c-4988-a6be-37e3651dea60 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:14.442927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:14.454809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:14.456020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:14.456938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:14.459745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 479/956] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:14.468568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:14.487688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f5374-885e-4e55-9988-ccf8a7cad635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:14.488816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f5374-885e-4e55-9988-ccf8a7cad635 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:14.489309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f5374-885e-4e55-9988-ccf8a7cad635 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:14.489755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f5374-885e-4e55-9988-ccf8a7cad635 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/ce1444d8c66045389eac37f8a7309378/roles/0b69eefec9d249b39b3ef87c6de99c50` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:14.492741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f5374-885e-4e55-9988-ccf8a7cad635 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=ce1444d8c66045389eac37f8a7309378, role_id=0b69eefec9d249b39b3ef87c6de99c50)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:14.494343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f5374-885e-4e55-9988-ccf8a7cad635 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:14.519267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 478/957] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:06:14 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/ce1444d8c66045389eac37f8a7309378/roles/0b69eefec9d249b39b3ef87c6de99c50 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:06:14.656019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd9bbb-fb5e-4b19-a006-8c2cdc27471f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:14.686619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b178-a940-49f7-9ed6-e74c708255f4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:14.686980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b178-a940-49f7-9ed6-e74c708255f4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:14.687178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b178-a940-49f7-9ed6-e74c708255f4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:14.691962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 479/958] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:14.698985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e929-571d-40a6-ac3e-c4884875a699 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:14.699733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e929-571d-40a6-ac3e-c4884875a699 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:14.700146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e929-571d-40a6-ac3e-c4884875a699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:14.719339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:14.719339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:14.719339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:14.719339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:14.720008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:14.720008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:14.721411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:14.728536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 480/959] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:14.731766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-27b7e929-571d-40a6-ac3e-c4884875a699 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:14.733389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-27b7e929-571d-40a6-ac3e-c4884875a699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:14.734651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:14.809673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:14.845226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 480/960] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:14.862091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:14.942067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91f1b8-d7f2-4cf6-ad1f-160cde85f1b3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:15.008288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:15.009262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:15.009764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:15.010255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:15.011728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:15.012054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:15.014668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:15.026350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 481/961] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:15.033153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7e929-571d-40a6-ac3e-c4884875a699 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:15.145688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7e929-571d-40a6-ac3e-c4884875a699 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:15.181403 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ef89-4153-46c5-a4e0-944a9e35bb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:15.182020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ef89-4153-46c5-a4e0-944a9e35bb56 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:15.182298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ef89-4153-46c5-a4e0-944a9e35bb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:15.182570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ef89-4153-46c5-a4e0-944a9e35bb56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:15.183312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702ef89-4153-46c5-a4e0-944a9e35bb56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:15.184701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702ef89-4153-46c5-a4e0-944a9e35bb56 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:15.206859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 481/962] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:06:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:15.304495 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:15.436228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf7a09-2712-44f4-869f-22e4294a8bca None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:15.466133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7609-707a-4d4b-96ff-0a76d2cdba2b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:15.466433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7609-707a-4d4b-96ff-0a76d2cdba2b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:15.466515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7609-707a-4d4b-96ff-0a76d2cdba2b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:15.468376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 482/963] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:15.474055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98798d9c-a080-4bbe-81cb-9da38da975ad None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:15.474325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98798d9c-a080-4bbe-81cb-9da38da975ad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:15.474595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98798d9c-a080-4bbe-81cb-9da38da975ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:15.477828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:15.490630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:15.491272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:15.491558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:15.491843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:15.492463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:15.493685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-98798d9c-a080-4bbe-81cb-9da38da975ad None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:15.494134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:15.494879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-98798d9c-a080-4bbe-81cb-9da38da975ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:15.497249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING keystone.server.flask.common [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 12:06:15.515651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.sql.core [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 12:06:15.515651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 12:06:15.515651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [parameters: {'user_id': 'dcf061975c6d435284a3b41382826df7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 12:06:15.515651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81487) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application conn.query(q) Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result.read() Aug 31 12:06:15.524947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 12:06:15.526672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application next(self.gen) Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application yield resource Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application next(self.gen) Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise self.value Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application session.commit() Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 12:06:15.528782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.session.flush() Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application compat.raise_( Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 12:06:15.530420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 12:06:15.538134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application conn.query(q) Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application result.read() Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 12:06:15.541333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dcf061975c6d435284a3b41382826df7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:15.544744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 12:06:15.551415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 12:06:15.555009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 12:06:15.555009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 12:06:15.555009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 12:06:15.555009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 12:06:15.555009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 12:06:15.555009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:15.555009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 482/964] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 31 12:06:15 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:15.562232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:15.562413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:15.580785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 483/965] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:15.702884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc360e27-554b-49da-8c3d-a5a9b26a987e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:15.744867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:15.746312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:15.746980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:15.747474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:15.755975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:15.757954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:06:15.772170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:15.773220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 12:06:15.774181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:15.776612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 483/966] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:06:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:15.782897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98798d9c-a080-4bbe-81cb-9da38da975ad None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:15.862460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:15.920976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98798d9c-a080-4bbe-81cb-9da38da975ad None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:15.947981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:15.961375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68e2dc-3b59-4113-8fb5-5502b840dd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:15.962054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e2dc-3b59-4113-8fb5-5502b840dd0b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:15.962253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e2dc-3b59-4113-8fb5-5502b840dd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:15.962522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e2dc-3b59-4113-8fb5-5502b840dd0b None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:15.963532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68e2dc-3b59-4113-8fb5-5502b840dd0b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:15.963532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68e2dc-3b59-4113-8fb5-5502b840dd0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:15.965192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68e2dc-3b59-4113-8fb5-5502b840dd0b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:16.015271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 484/967] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 31 12:06:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:16.016945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bc741-1aa2-43f5-9f47-2812f0ca66b6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:16.059614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb9700-cdf0-4177-842c-a1b7059b1422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:16.060283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb9700-cdf0-4177-842c-a1b7059b1422 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:16.060567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb9700-cdf0-4177-842c-a1b7059b1422 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:16.060839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb9700-cdf0-4177-842c-a1b7059b1422 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:16.061507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb9700-cdf0-4177-842c-a1b7059b1422 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:16.063036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb9700-cdf0-4177-842c-a1b7059b1422 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:16.085549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 484/968] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:06:15 2022] POST /identity/v3/services => generated 242 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:06:16.628831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8caa7-3175-4615-ba43-fb5461f1ac4a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:16.629439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8caa7-3175-4615-ba43-fb5461f1ac4a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:16.629439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8caa7-3175-4615-ba43-fb5461f1ac4a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:16.630809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 485/969] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:16.637211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba642ff5-9575-45bf-a044-66f62aee04f4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:16.637912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba642ff5-9575-45bf-a044-66f62aee04f4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:16.637912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba642ff5-9575-45bf-a044-66f62aee04f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:16.656923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ba642ff5-9575-45bf-a044-66f62aee04f4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:16.658480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ba642ff5-9575-45bf-a044-66f62aee04f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:16.763357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 485/970] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:17.023269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8caa7-3175-4615-ba43-fb5461f1ac4a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.053655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:17.133228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8caa7-3175-4615-ba43-fb5461f1ac4a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:17.168648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e02ff-1467-4a66-8695-02ce140fa92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:17.169175 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e02ff-1467-4a66-8695-02ce140fa92e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.169448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e02ff-1467-4a66-8695-02ce140fa92e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.169731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e02ff-1467-4a66-8695-02ce140fa92e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.180685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e02ff-1467-4a66-8695-02ce140fa92e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-032e02ff-1467-4a66-8695-02ce140fa92e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:17.185618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:17.186561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:17.187858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:17.191627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 486/971] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:17.192895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba642ff5-9575-45bf-a044-66f62aee04f4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.227009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:17.294097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe2a3-612b-44f2-bea8-c11cbe94dd6a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.294434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe2a3-612b-44f2-bea8-c11cbe94dd6a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.294656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe2a3-612b-44f2-bea8-c11cbe94dd6a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.296808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 487/972] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:17.301929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f3f6-a3c3-43aa-a248-596d2f468f05 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.302215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f3f6-a3c3-43aa-a248-596d2f468f05 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.302494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f3f6-a3c3-43aa-a248-596d2f468f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.308460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba642ff5-9575-45bf-a044-66f62aee04f4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:17.322615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c898f3f6-a3c3-43aa-a248-596d2f468f05 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:17.324165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c898f3f6-a3c3-43aa-a248-596d2f468f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:17.350445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:17.351400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.351906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.352368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.353285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:17.353750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:17.355426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:17.364798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 486/973] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:17.370848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.397047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 488/974] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:17.471624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:17.479371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07ea64-8a7e-4d4f-8bf8-f6e80a59fe31 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:17.516803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:17.517446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.517719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.517986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.529870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:17.531417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:17.547532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:17.548724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:17.549658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:17.549658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:17.549658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:17.551598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 487/975] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:17.554637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898f3f6-a3c3-43aa-a248-596d2f468f05 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.585482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:17.666574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.714138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898f3f6-a3c3-43aa-a248-596d2f468f05 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:17.753654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:17.754188 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.754465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.754761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.755418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:17.755721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:17.757142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:17.775916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 489/976] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:17.777373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.800402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e822ec-4862-41b4-bb19-266e9078caa4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:17.836673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:17.837366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.837574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.837831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.838408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:17.839884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:17.849650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:17.850927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:17.851980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 488/977] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:17.857033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.873914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:17.875871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:17.901753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dce32-c6ed-4edc-ad55-dc6eee173ba5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:17.941231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:17.941960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:17.942300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:17.942644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:17.952335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:17.953883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:06:17.966375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:17.967824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:17.974243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 490/978] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:17.975715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:17.983289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa9c53-62f2-436d-89ea-efd3e43a25d3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.030434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.030946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.031218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.031568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.032134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:18.032457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.033919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:18.045813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 489/979] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:18.047104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:18.097367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7c89-f3f6-4fa9-8942-d120050419f4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.130794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:18.136813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc0ee-8002-4386-959b-4d611beb18f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.137455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc0ee-8002-4386-959b-4d611beb18f6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.137717 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc0ee-8002-4386-959b-4d611beb18f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.138001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc0ee-8002-4386-959b-4d611beb18f6 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.138733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc0ee-8002-4386-959b-4d611beb18f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:18.139024 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc0ee-8002-4386-959b-4d611beb18f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.140510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc0ee-8002-4386-959b-4d611beb18f6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:18.149417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:18.158105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85826c7d-c489-447e-94f5-c273d97abd31 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.159861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 491/980] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 31 12:06:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:18.160908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa675dfd-b388-4346-821d-fd780afe5312 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.161194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa675dfd-b388-4346-821d-fd780afe5312 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.161470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa675dfd-b388-4346-821d-fd780afe5312 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.162702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 492/981] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:18.165362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa675dfd-b388-4346-821d-fd780afe5312 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:18.198314 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ebba9-bf65-46c7-b617-86fe58ad4248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.198971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ebba9-bf65-46c7-b617-86fe58ad4248 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.199464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ebba9-bf65-46c7-b617-86fe58ad4248 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.199745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ebba9-bf65-46c7-b617-86fe58ad4248 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.200380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ebba9-bf65-46c7-b617-86fe58ad4248 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:18.200685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ebba9-bf65-46c7-b617-86fe58ad4248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.202078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ebba9-bf65-46c7-b617-86fe58ad4248 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:18.212463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 490/982] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:18.212463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415be464-574a-4e9e-a667-89a19465c4d3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.212463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415be464-574a-4e9e-a667-89a19465c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.212463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415be464-574a-4e9e-a667-89a19465c4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.228265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-415be464-574a-4e9e-a667-89a19465c4d3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:18.229473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-415be464-574a-4e9e-a667-89a19465c4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:18.270338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa675dfd-b388-4346-821d-fd780afe5312 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.301671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 491/983] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:18.302223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415be464-574a-4e9e-a667-89a19465c4d3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:18.309738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c907-b53e-4f51-abfd-5383e6f95e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.310367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c907-b53e-4f51-abfd-5383e6f95e96 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.310676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c907-b53e-4f51-abfd-5383e6f95e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.311085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c907-b53e-4f51-abfd-5383e6f95e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.379154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:18.389337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:18.397393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5c907-b53e-4f51-abfd-5383e6f95e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.398842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5c907-b53e-4f51-abfd-5383e6f95e96 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:18.419977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415be464-574a-4e9e-a667-89a19465c4d3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.456248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.456866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.457140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.457401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.458325 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] RBAC: Adding query filter params (interface=admin, service_id=f60a1a1ceb70478fb835a3311cad73ae, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:18.458458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.459807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:18.469899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 492/984] 10.222.0.27 () {56 vars in 1249 bytes} [Wed Aug 31 12:06:18 2022] GET /identity/v3/endpoints?service_id=f60a1a1ceb70478fb835a3311cad73ae&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:18.470988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:18.522982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 493/985] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:18 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 12:06:18.528753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c907-b53e-4f51-abfd-5383e6f95e96 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:18.582053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22740c07-8587-4061-a682-88fa9ca3b6d6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.622463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:18.629455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a8d3-3504-4093-817c-3a3b4259a285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.630106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a8d3-3504-4093-817c-3a3b4259a285 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.630389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a8d3-3504-4093-817c-3a3b4259a285 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.630671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a8d3-3504-4093-817c-3a3b4259a285 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.631364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3a8d3-3504-4093-817c-3a3b4259a285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.632822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3a8d3-3504-4093-817c-3a3b4259a285 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:18.639567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:18.650794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 493/986] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:06:18 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:06:18.650794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a8d3-3504-4093-817c-3a3b4259a285 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:18.660499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c907-b53e-4f51-abfd-5383e6f95e96 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.710584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.711606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.712038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.712572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.726457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.728191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:18.750181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:18.751425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:18.752491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:18.752491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:18.752491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:18.752965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 494/987] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:18.758633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:18.797446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a8d3-3504-4093-817c-3a3b4259a285 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.853654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.854215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.854485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.854747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.855425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.856905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:18.875009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:18.876975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:18.878648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 494/988] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:18.884008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:18.939652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796735-d488-48c5-b48f-b92021bba58c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:18.961973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:18.984485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:18.985354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:18.986351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:18.986351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:18.986619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:18.986840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:18.988300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:19.009067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 495/989] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:19.019188 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:19.021474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17776b23-a9d2-4600-984e-07fa1fad5267 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:19.034124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:19.065561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:19.066489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:19.066926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:19.067390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:19.068391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:19.068876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:19.071180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:19.078763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 495/990] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:06:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:19.084708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:19.153825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f90eb0-0370-4eed-bcad-8fc3c27541c7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:19.210122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:19.211100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:19.211594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:19.212048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:19.223906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:19.226386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:19.240950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63016-f541-49e3-b5e4-9960546b1796 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:19.245903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:19.247803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:19.249477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 496/991] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:19.256844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:19.283521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:19.299619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:19.300330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:19.300751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:19.301198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:19.302164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:19.302609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:19.304344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:19.316377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 496/992] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:06:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:19.317887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:19.350841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:19.393437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e21056-a277-44d7-a30a-bff78fa42682 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:19.437075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:19.437876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:19.438330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:19.438778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:19.439959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:19.440462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:19.442770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:19.450192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039fe4a-511b-4369-b9c2-22c49b2f853d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:19.469854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 497/993] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:19.471376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:19.511001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:19.511680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:19.512132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:19.512552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:19.513747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] RBAC: Adding query filter params (interface=public, service_id=961001c61e23470b8e662b97982aafd5, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:19.514248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:19.516449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:19.530546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 497/994] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 31 12:06:19 2022] GET /identity/v3/endpoints?service_id=961001c61e23470b8e662b97982aafd5&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:19.532352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:19.602758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:19.621165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9495a-eab6-4e0b-b8ae-dffc0a04fc95 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:19.658206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa758fd0-9e76-4613-854a-c22d9c14d889 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:19.668506 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:19.680423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013c492-285c-44d6-adf7-ff883e3e9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:19.681415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c492-285c-44d6-adf7-ff883e3e9539 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:19.682533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c492-285c-44d6-adf7-ff883e3e9539 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:19.683076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c492-285c-44d6-adf7-ff883e3e9539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:19.684643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013c492-285c-44d6-adf7-ff883e3e9539 None admin] RBAC: Adding query filter params (role.id=0b69eefec9d249b39b3ef87c6de99c50, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=ce1444d8c66045389eac37f8a7309378) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:19.685176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013c492-285c-44d6-adf7-ff883e3e9539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:19.687915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013c492-285c-44d6-adf7-ff883e3e9539 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:19.697340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13e712-fcc3-4064-b695-c7bbd33e06e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:19.697961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13e712-fcc3-4064-b695-c7bbd33e06e0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:19.698238 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13e712-fcc3-4064-b695-c7bbd33e06e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:19.698508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13e712-fcc3-4064-b695-c7bbd33e06e0 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:19.699153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13e712-fcc3-4064-b695-c7bbd33e06e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:19.700678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13e712-fcc3-4064-b695-c7bbd33e06e0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:19.707520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 498/995] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:19 2022] GET /identity/v3/role_assignments?user.id=ce1444d8c66045389eac37f8a7309378&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=0b69eefec9d249b39b3ef87c6de99c50 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:19.711267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 498/996] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:06:19 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:06:20.632660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dce60-1c53-414f-839b-fd16c7d23e28 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:20.633500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dce60-1c53-414f-839b-fd16c7d23e28 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:20.633500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dce60-1c53-414f-839b-fd16c7d23e28 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:20.634793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 499/997] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:20.641813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8b68b-2dff-46d5-9338-0a3acb41c15c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:20.642313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8b68b-2dff-46d5-9338-0a3acb41c15c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:20.642945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8b68b-2dff-46d5-9338-0a3acb41c15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:20.666520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f6e8b68b-2dff-46d5-9338-0a3acb41c15c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:20.667954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f6e8b68b-2dff-46d5-9338-0a3acb41c15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:20.771436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 499/998] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:21.046191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dce60-1c53-414f-839b-fd16c7d23e28 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:21.156487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dce60-1c53-414f-839b-fd16c7d23e28 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:21.192171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:21.192807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:21.193082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:21.193333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:21.193914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:21.195331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:21.204756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:21.205808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:21.206693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 500/999] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 31 12:06:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:21.211827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8b68b-2dff-46d5-9338-0a3acb41c15c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:21.267768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:21.330115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8b68b-2dff-46d5-9338-0a3acb41c15c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:21.369394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:21.370040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:21.370364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:21.370669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:21.371928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:21.372279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:21.373779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:21.379594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 500/1000] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:21.385917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:21.485209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:21.555613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c3513-4d41-4ed3-86ad-36ed0bbbce7b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:21.595531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c1ba1-be3f-45fa-aba9-42d304d3df7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:21.596182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1ba1-be3f-45fa-aba9-42d304d3df7c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:21.596476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1ba1-be3f-45fa-aba9-42d304d3df7c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:21.596758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1ba1-be3f-45fa-aba9-42d304d3df7c None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:21.597701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c1ba1-be3f-45fa-aba9-42d304d3df7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:21.598009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c1ba1-be3f-45fa-aba9-42d304d3df7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:21.599458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c1ba1-be3f-45fa-aba9-42d304d3df7c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:21.605458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 501/1001] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 31 12:06:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:21.612151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:21.797729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20654613-fd45-41e7-ab02-b6a1a6ab1276 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:21.818227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:21.829319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115969f-6dc2-4c99-a397-264154b529a7 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:21.829899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115969f-6dc2-4c99-a397-264154b529a7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:21.830284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115969f-6dc2-4c99-a397-264154b529a7 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:21.832975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 502/1002] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:21.840009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749f235-49e0-4328-9a4d-8f0c0b6d239e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:21.840364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749f235-49e0-4328-9a4d-8f0c0b6d239e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:21.840763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749f235-49e0-4328-9a4d-8f0c0b6d239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:21.851867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57a7a4-8bd1-4ce9-822e-60007e4cd265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:21.852479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a7a4-8bd1-4ce9-822e-60007e4cd265 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:21.852752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a7a4-8bd1-4ce9-822e-60007e4cd265 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:21.853021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a7a4-8bd1-4ce9-822e-60007e4cd265 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:21.853691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57a7a4-8bd1-4ce9-822e-60007e4cd265 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:21.855193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57a7a4-8bd1-4ce9-822e-60007e4cd265 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:21.861344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9749f235-49e0-4328-9a4d-8f0c0b6d239e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:21.862565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9749f235-49e0-4328-9a4d-8f0c0b6d239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:21.877362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 501/1003] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:06:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:06:21.979346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 503/1004] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:22.074390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def494bc-0b62-44a2-a1e2-c53cce4f8bd4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:22.074829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def494bc-0b62-44a2-a1e2-c53cce4f8bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:22.075117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def494bc-0b62-44a2-a1e2-c53cce4f8bd4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:22.076608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 502/1005] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:22.082796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831790c1-3de5-4521-aa39-102f1dd76252 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:22.083171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831790c1-3de5-4521-aa39-102f1dd76252 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:22.083611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831790c1-3de5-4521-aa39-102f1dd76252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:22.105347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-831790c1-3de5-4521-aa39-102f1dd76252 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:22.106686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-831790c1-3de5-4521-aa39-102f1dd76252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:22.171018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:22.212736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 504/1006] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:22.363701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def494bc-0b62-44a2-a1e2-c53cce4f8bd4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:22.482067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def494bc-0b62-44a2-a1e2-c53cce4f8bd4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:22.523678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:22.524480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:22.524874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:22.525299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] PATH_INFO: `/v3/services/image` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:22.526021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:22.527873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:22.537660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:22.538716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:22.540065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:22.545827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 503/1007] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 31 12:06:22 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:22.546003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831790c1-3de5-4521-aa39-102f1dd76252 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:22.546710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:22.586817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:22.665430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f46281-438a-4e06-8d9e-53224a2057cc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:22.694985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831790c1-3de5-4521-aa39-102f1dd76252 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:22.712989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:22.713975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:22.714480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:22.714928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:22.715696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:22.717585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:22.728128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:22.729189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:22.730107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 504/1008] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:22.735343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:22.760835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:22.762547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:22.763037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:22.763567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:22.764732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] RBAC: Adding query filter params (name=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:22.765285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:22.767970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:22.782741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 505/1009] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:22.784548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:22.853165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:22.860129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:22.864373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe179f-8b25-4e87-8592-099e3229f1c0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:22.909782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:22.910589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:22.911042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:22.911471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:22.912467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:22.912893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:22.914647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:22.923348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 505/1010] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:06:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:22.927006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:22.942009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07a8cc-5cf4-4d6f-8377-9eaf42e6dedb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:22.980683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:22.981295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:22.981573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:22.981830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:22.982461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] RBAC: Adding query filter params (type=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:22.982763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:22.984223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:22.989493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 506/1011] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:23.069435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1ad6f-0c84-4bf0-8e69-c18f9d502881 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:23.110668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fca9ae-9282-42c3-8c8b-6a3b141697be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:23.111593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fca9ae-9282-42c3-8c8b-6a3b141697be None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:23.111998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fca9ae-9282-42c3-8c8b-6a3b141697be None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:23.112422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fca9ae-9282-42c3-8c8b-6a3b141697be None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:23.113286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fca9ae-9282-42c3-8c8b-6a3b141697be None admin] RBAC: Adding query filter params (type=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:23.113709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fca9ae-9282-42c3-8c8b-6a3b141697be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:23.115290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fca9ae-9282-42c3-8c8b-6a3b141697be None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:23.121927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 506/1012] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 31 12:06:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:23.128044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:23.287961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37952b-d074-4df2-bb51-78f88cc73975 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:23.290316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:23.340286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be2773-89ac-49d6-abf8-939c3b1663c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:23.341162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2773-89ac-49d6-abf8-939c3b1663c3 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:23.341478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2773-89ac-49d6-abf8-939c3b1663c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:23.341744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2773-89ac-49d6-abf8-939c3b1663c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:23.342430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be2773-89ac-49d6-abf8-939c3b1663c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:23.343850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be2773-89ac-49d6-abf8-939c3b1663c3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:23.366294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 507/1013] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:06:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:24.006692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61bfe-af73-4e18-bf79-7dc16abea89d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:24.007400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61bfe-af73-4e18-bf79-7dc16abea89d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:24.007400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61bfe-af73-4e18-bf79-7dc16abea89d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:24.008829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 507/1014] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:24.014471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c44ff-aa83-4f9f-acd2-397f33f5ab96 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:24.014960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c44ff-aa83-4f9f-acd2-397f33f5ab96 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:24.015039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c44ff-aa83-4f9f-acd2-397f33f5ab96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:24.033916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cf8c44ff-aa83-4f9f-acd2-397f33f5ab96 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:24.035112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cf8c44ff-aa83-4f9f-acd2-397f33f5ab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:24.118163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 508/1015] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:24 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:24.395491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b61bfe-af73-4e18-bf79-7dc16abea89d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:24.427633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:24.510187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b61bfe-af73-4e18-bf79-7dc16abea89d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:24.545529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d657b12-3a05-4705-9988-88991b3c141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:24.546164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d657b12-3a05-4705-9988-88991b3c141e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:24.546438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d657b12-3a05-4705-9988-88991b3c141e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:24.546703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d657b12-3a05-4705-9988-88991b3c141e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:24.547846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d657b12-3a05-4705-9988-88991b3c141e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:24.549285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d657b12-3a05-4705-9988-88991b3c141e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:24.551037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 508/1016] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:24.556783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c44ff-aa83-4f9f-acd2-397f33f5ab96 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:24.646530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:24.662909 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c44ff-aa83-4f9f-acd2-397f33f5ab96 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:24.697448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f93e84-2f31-4337-9753-e76e2fe9e1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:24.698055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93e84-2f31-4337-9753-e76e2fe9e1a5 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:24.698332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93e84-2f31-4337-9753-e76e2fe9e1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:24.698765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93e84-2f31-4337-9753-e76e2fe9e1a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:24.699621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f93e84-2f31-4337-9753-e76e2fe9e1a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:24.701061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f93e84-2f31-4337-9753-e76e2fe9e1a5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:24.728514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 509/1017] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:06:24 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:24.939741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4379b-e4f4-4957-9714-0286b2a1e441 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:24.939925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4379b-e4f4-4957-9714-0286b2a1e441 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:24.940200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4379b-e4f4-4957-9714-0286b2a1e441 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:24.941589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 509/1018] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:24.947511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70b6fd-d4c9-4b86-b65f-2326f3a7d0de None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:24.947819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70b6fd-d4c9-4b86-b65f-2326f3a7d0de None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:24.948059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70b6fd-d4c9-4b86-b65f-2326f3a7d0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:24.967608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2c70b6fd-d4c9-4b86-b65f-2326f3a7d0de None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:24.968826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2c70b6fd-d4c9-4b86-b65f-2326f3a7d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:25.045440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 510/1019] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:24 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:25.310313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4379b-e4f4-4957-9714-0286b2a1e441 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:25.422625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:25.432488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4379b-e4f4-4957-9714-0286b2a1e441 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:25.445767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e61c-f465-4734-9fd8-ed4397173f56 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:25.446564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e61c-f465-4734-9fd8-ed4397173f56 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:25.446926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e61c-f465-4734-9fd8-ed4397173f56 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:25.448388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 511/1020] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:25.454491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd31ae-38da-414e-aa4d-5a8a7883b079 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:25.454828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd31ae-38da-414e-aa4d-5a8a7883b079 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:25.455040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd31ae-38da-414e-aa4d-5a8a7883b079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:25.474650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-15dd31ae-38da-414e-aa4d-5a8a7883b079 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:25.475911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-15dd31ae-38da-414e-aa4d-5a8a7883b079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:25.476288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135770b-3ccd-4a43-9513-e7d62da75dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:25.476908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135770b-3ccd-4a43-9513-e7d62da75dc6 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:25.477181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135770b-3ccd-4a43-9513-e7d62da75dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:25.477452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135770b-3ccd-4a43-9513-e7d62da75dc6 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:25.478118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135770b-3ccd-4a43-9513-e7d62da75dc6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:25.479554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135770b-3ccd-4a43-9513-e7d62da75dc6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:25.507284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 510/1021] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:06:25 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:25.575672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:25.593560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 512/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:25.869312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135770b-3ccd-4a43-9513-e7d62da75dc6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:25.981213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135770b-3ccd-4a43-9513-e7d62da75dc6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:26.016408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576cd00-631c-4e68-b818-ef817f553d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:26.017056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd00-631c-4e68-b818-ef817f553d09 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:26.017318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd00-631c-4e68-b818-ef817f553d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:26.017580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd00-631c-4e68-b818-ef817f553d09 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:26.029173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576cd00-631c-4e68-b818-ef817f553d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-5576cd00-631c-4e68-b818-ef817f553d09 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:26.034105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:26.037734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:26.043501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 511/1023] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 31 12:06:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:26.049061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd31ae-38da-414e-aa4d-5a8a7883b079 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:26.171331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd31ae-38da-414e-aa4d-5a8a7883b079 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:26.212615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:26.213241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:26.213517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:26.213783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:26.215349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:26.215663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:26.217099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:26.227033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 513/1024] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 31 12:06:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:26.233867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576cd00-631c-4e68-b818-ef817f553d09 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:26.252989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:26.388854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576cd00-631c-4e68-b818-ef817f553d09 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:26.426118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:26.426664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:26.426937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:26.427204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:26.438456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:26.439949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:26.454435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:26.456235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:26.457950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:26.457950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:26.457950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:26.457950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 512/1025] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:26.464333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:26.540752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:26.596496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e681-1525-4c42-9557-37a90bf81f2f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:26.634334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34fc3-7ffa-4000-a13e-8128d0fc4f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:26.635341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34fc3-7ffa-4000-a13e-8128d0fc4f5b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:26.635810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34fc3-7ffa-4000-a13e-8128d0fc4f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:26.636595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34fc3-7ffa-4000-a13e-8128d0fc4f5b None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:26.637463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34fc3-7ffa-4000-a13e-8128d0fc4f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:26.637957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34fc3-7ffa-4000-a13e-8128d0fc4f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:26.639551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34fc3-7ffa-4000-a13e-8128d0fc4f5b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:26.653048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 514/1026] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:26.658518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:26.743565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:26.754681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e93c4-d82c-4fe6-9e32-fbae1f69cfce None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:26.755005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e93c4-d82c-4fe6-9e32-fbae1f69cfce None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:26.755214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e93c4-d82c-4fe6-9e32-fbae1f69cfce None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:26.756721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 515/1027] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:26.761710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38292e0e-bc0c-4ff2-aa45-862b8aeb61e7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:26.761990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38292e0e-bc0c-4ff2-aa45-862b8aeb61e7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:26.762259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38292e0e-bc0c-4ff2-aa45-862b8aeb61e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:26.780653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-38292e0e-bc0c-4ff2-aa45-862b8aeb61e7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:26.781835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-38292e0e-bc0c-4ff2-aa45-862b8aeb61e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:26.796355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008aec9c-f33d-443d-b68c-5dd62e8b41ca None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:26.835051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:26.835714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:26.835985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:26.836251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:26.846421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:26.848237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:06:26.860298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:26.868142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:26.869629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 513/1028] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:06:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:26.875848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 516/1029] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:26.878102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:27.004691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f531d6-6956-4db8-b3e7-3b179d25b9e5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:27.052959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:27.068648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:27.069625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:27.070082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:27.070521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:27.071822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:27.072326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:27.077474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:27.114294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 514/1030] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:06:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:27.116190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38292e0e-bc0c-4ff2-aa45-862b8aeb61e7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:27.129641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:27.171909 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:27.252402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38292e0e-bc0c-4ff2-aa45-862b8aeb61e7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:27.264502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672198b4-3e1e-490e-ab64-afb091740842 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:27.310740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f2cff-55d8-447b-857c-d43f1ab01b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:27.311836 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f2cff-55d8-447b-857c-d43f1ab01b92 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:27.312265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f2cff-55d8-447b-857c-d43f1ab01b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:27.312722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f2cff-55d8-447b-857c-d43f1ab01b92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:27.322262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ae548-a550-45c9-a41e-c6a7571bf0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:27.323305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ae548-a550-45c9-a41e-c6a7571bf0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:27.324303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ae548-a550-45c9-a41e-c6a7571bf0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:27.324808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ae548-a550-45c9-a41e-c6a7571bf0c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:27.326874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ae548-a550-45c9-a41e-c6a7571bf0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:27.328716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ae548-a550-45c9-a41e-c6a7571bf0c2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:27.336595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 515/1031] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:27.338136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ae548-a550-45c9-a41e-c6a7571bf0c2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:27.415170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f2cff-55d8-447b-857c-d43f1ab01b92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:27.416997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f2cff-55d8-447b-857c-d43f1ab01b92 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:27.417812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:27.470830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:27.486092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ae548-a550-45c9-a41e-c6a7571bf0c2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:27.531711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b2a8a-7a01-408d-b9b7-662959bcd43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:27.532342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b2a8a-7a01-408d-b9b7-662959bcd43b None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:27.532547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b2a8a-7a01-408d-b9b7-662959bcd43b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:27.532817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b2a8a-7a01-408d-b9b7-662959bcd43b None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:27.533478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b2a8a-7a01-408d-b9b7-662959bcd43b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:27.535064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b2a8a-7a01-408d-b9b7-662959bcd43b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:27.546287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 517/1032] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:06:27.549318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f2cff-55d8-447b-857c-d43f1ab01b92 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:27.559706 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 516/1033] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:06:27 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:06:27.561407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf07e7-d314-4f6a-8c2e-232fec0abb64 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:27.561910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf07e7-d314-4f6a-8c2e-232fec0abb64 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:27.562386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf07e7-d314-4f6a-8c2e-232fec0abb64 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:27.564014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 517/1034] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:27.573602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7cd7-bed5-4211-aa7c-86a997e67d80 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:27.574883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7cd7-bed5-4211-aa7c-86a997e67d80 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:27.575159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7cd7-bed5-4211-aa7c-86a997e67d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:27.595339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-621d7cd7-bed5-4211-aa7c-86a997e67d80 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:27.596588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-621d7cd7-bed5-4211-aa7c-86a997e67d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:27.679785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 518/1035] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:27.733755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f2cff-55d8-447b-857c-d43f1ab01b92 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:27.798754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:27.799356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:27.799634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:27.799910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:27.812133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:27.813613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:27.822301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:27.832277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:27.833298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:27.834225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:27.834225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:27.834225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:27.834225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 518/1036] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:27.840669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7cd7-bed5-4211-aa7c-86a997e67d80 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:27.895710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:27.963078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7cd7-bed5-4211-aa7c-86a997e67d80 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:28.002469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:28.003082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:28.003392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:28.003664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:28.004302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:28.004601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:28.005986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:28.023268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 519/1037] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:28.026653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:28.067016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:28.189699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d4974-8657-4c6a-a34a-b1ef3618427a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:28.218740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:28.227124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e8c7e-896e-4c75-a6b9-6f1120879fd7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:28.244403 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:28.245051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:28.245575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:28.245701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:28.256637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:28.258649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:28.276362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:28.277966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:28.280056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 519/1038] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:28.283705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:28.284512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:28.285634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:28.286105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:28.286559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] PATH_INFO: `/v3/services/image` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:28.287598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:28.291954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:28.311048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:28.312202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:28.313063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:28.314818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 520/1039] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 31 12:06:28 2022] GET /identity/v3/services/image => generated 86 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:28.322702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:28.383621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:28.445929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309cc92-7055-42ae-9470-ce89b439675d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:28.467416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ffa19-8ff1-4e91-9edf-c298115e2531 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:28.506573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:28.507085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:28.507206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:28.507479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:28.507747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:28.508130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:28.508365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:28.508598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:28.508703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:28.509080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:28.510078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:28.510343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:28.510947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:28.514351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:28.515954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 521/1040] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:28.521802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:28.545137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 520/1041] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:28.546807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:28.564953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:28.661903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fe2d7-862c-4be9-93e7-dd82a5ca55d6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:28.684085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660223a0-4534-42aa-b4eb-b53a2c48d668 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:28.705554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:28.715319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:28.716137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:28.716489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:28.716833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:28.717647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:28.718038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:28.719916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:28.727430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 522/1042] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:28.728717 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:28.729567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:28.729879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:28.730149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:28.730879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] RBAC: Adding query filter params (role.id=0b69eefec9d249b39b3ef87c6de99c50, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=0b113316bc3348b5a12c1d7b616bf4c9) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:28.731179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:28.732684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:28.736183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:28.751267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 521/1043] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:28 2022] GET /identity/v3/role_assignments?user.id=0b113316bc3348b5a12c1d7b616bf4c9&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=0b69eefec9d249b39b3ef87c6de99c50 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:28.907612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:28.925536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300dc7c-0550-4a4a-85d4-ef843061a8e6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:28.982763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0fe27-3f19-478a-9d54-7c4fb831d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:28.983605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0fe27-3f19-478a-9d54-7c4fb831d902 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:28.983967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0fe27-3f19-478a-9d54-7c4fb831d902 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:28.984308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0fe27-3f19-478a-9d54-7c4fb831d902 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:28.985360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0fe27-3f19-478a-9d54-7c4fb831d902 None admin] RBAC: Adding query filter params (interface=public, service_id=b3164cf229654496b82ee3ed189ed57a, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:28.985873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0fe27-3f19-478a-9d54-7c4fb831d902 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:28.988212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0fe27-3f19-478a-9d54-7c4fb831d902 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:28.997047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 523/1044] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 31 12:06:28 2022] GET /identity/v3/endpoints?service_id=b3164cf229654496b82ee3ed189ed57a&interface=public®ion_id=RegionOne => generated 194 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:29.005948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:29.164082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4638-ae14-4fe4-b9ac-cfd5515c2cbf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:29.216858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3a441-ee7e-45ee-8e75-a1ba05da5fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:29.217812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3a441-ee7e-45ee-8e75-a1ba05da5fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:29.218094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3a441-ee7e-45ee-8e75-a1ba05da5fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:29.218371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3a441-ee7e-45ee-8e75-a1ba05da5fc1 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:29.218992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3a441-ee7e-45ee-8e75-a1ba05da5fc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:29.223382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3a441-ee7e-45ee-8e75-a1ba05da5fc1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:29.233849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 522/1045] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:06:29 2022] GET /identity/v3/services => generated 1957 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:06:29.685264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6222-1cf6-4169-9540-d116c7bec944 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:29.685776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6222-1cf6-4169-9540-d116c7bec944 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:29.686089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6222-1cf6-4169-9540-d116c7bec944 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:29.688050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 524/1046] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:29.693767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a9694-a381-4488-87c2-51464d818c4a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:29.694047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a9694-a381-4488-87c2-51464d818c4a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:29.694321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a9694-a381-4488-87c2-51464d818c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:29.716886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d67a9694-a381-4488-87c2-51464d818c4a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:29.718794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d67a9694-a381-4488-87c2-51464d818c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:29.800063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:29.842308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 523/1047] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:30.332422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e6222-1cf6-4169-9540-d116c7bec944 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:30.510108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e6222-1cf6-4169-9540-d116c7bec944 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:30.565121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf29e77-6bbb-4296-aeb9-0a24c69dc4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:30.566139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf29e77-6bbb-4296-aeb9-0a24c69dc4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:30.566579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf29e77-6bbb-4296-aeb9-0a24c69dc4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:30.567023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf29e77-6bbb-4296-aeb9-0a24c69dc4d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:30.585730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf29e77-6bbb-4296-aeb9-0a24c69dc4d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-faf29e77-6bbb-4296-aeb9-0a24c69dc4d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:30.594809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:30.595976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:06:30.597170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:30.599655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 525/1048] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:06:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:30.606168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a9694-a381-4488-87c2-51464d818c4a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:30.718550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a9694-a381-4488-87c2-51464d818c4a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:30.757354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ebfc3-6f82-492f-8ec8-ba3f62453263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:30.757966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ebfc3-6f82-492f-8ec8-ba3f62453263 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:30.758246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ebfc3-6f82-492f-8ec8-ba3f62453263 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:30.758516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ebfc3-6f82-492f-8ec8-ba3f62453263 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:30.759183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ebfc3-6f82-492f-8ec8-ba3f62453263 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:30.759528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ebfc3-6f82-492f-8ec8-ba3f62453263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:30.761078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ebfc3-6f82-492f-8ec8-ba3f62453263 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:30.776770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 524/1049] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:06:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:30.778721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf29e77-6bbb-4296-aeb9-0a24c69dc4d3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:30.797268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:30.906620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf29e77-6bbb-4296-aeb9-0a24c69dc4d3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:30.945870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab670a-f8d4-4b86-a1a8-9dee74a1faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:30.946396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab670a-f8d4-4b86-a1a8-9dee74a1faef None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:30.946677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab670a-f8d4-4b86-a1a8-9dee74a1faef None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:30.946940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab670a-f8d4-4b86-a1a8-9dee74a1faef None admin] PATH_INFO: `/v3/users/f6a74e43498847089af815b4cd6bfd1a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:30.960290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acae69-ce97-4e21-82d0-5512324d2760 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:30.960630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acae69-ce97-4e21-82d0-5512324d2760 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:30.960838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acae69-ce97-4e21-82d0-5512324d2760 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:30.962343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 525/1050] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:30.965710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab670a-f8d4-4b86-a1a8-9dee74a1faef None admin] RBAC: Authorizing `identity:get_user(user_id=f6a74e43498847089af815b4cd6bfd1a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:30.967569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab670a-f8d4-4b86-a1a8-9dee74a1faef None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:30.968138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb199c-da64-4381-bf43-b98116b2bbb1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:30.968424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb199c-da64-4381-bf43-b98116b2bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:30.968701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb199c-da64-4381-bf43-b98116b2bbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:30.970561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 526/1051] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:30 2022] GET /identity/v3/users/f6a74e43498847089af815b4cd6bfd1a => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:30.977345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab670a-f8d4-4b86-a1a8-9dee74a1faef None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:30.990662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6bdb199c-da64-4381-bf43-b98116b2bbb1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:30.992429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6bdb199c-da64-4381-bf43-b98116b2bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:31.029833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:31.090714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 526/1052] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:31.117496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:31.176916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab670a-f8d4-4b86-a1a8-9dee74a1faef None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:31.233225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded89ff2-fe00-429c-9161-673297bcab80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:31.234178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded89ff2-fe00-429c-9161-673297bcab80 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:31.234617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded89ff2-fe00-429c-9161-673297bcab80 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:31.235051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded89ff2-fe00-429c-9161-673297bcab80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:31.308636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed123a5-c184-4464-ae6c-78813e0619c0 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:31.308992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed123a5-c184-4464-ae6c-78813e0619c0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:31.309276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed123a5-c184-4464-ae6c-78813e0619c0 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:31.311007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 527/1053] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:31.316032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab69b-882b-49a2-90ef-21fcd0b26368 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:31.316387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab69b-882b-49a2-90ef-21fcd0b26368 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:31.316730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab69b-882b-49a2-90ef-21fcd0b26368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:31.326067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded89ff2-fe00-429c-9161-673297bcab80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:31.327614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded89ff2-fe00-429c-9161-673297bcab80 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:31.337148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c10ab69b-882b-49a2-90ef-21fcd0b26368 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:31.337977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c10ab69b-882b-49a2-90ef-21fcd0b26368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:31.411421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:31.414337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 528/1054] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:31.415187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ab69b-882b-49a2-90ef-21fcd0b26368 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:31.458350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 527/1055] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:06:31.465810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded89ff2-fe00-429c-9161-673297bcab80 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:31.535641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:31.539658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ab69b-882b-49a2-90ef-21fcd0b26368 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:31.579165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340493a-8102-465c-bda8-30f59b54947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:31.579844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340493a-8102-465c-bda8-30f59b54947a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:31.580128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340493a-8102-465c-bda8-30f59b54947a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:31.580400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340493a-8102-465c-bda8-30f59b54947a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:31.591549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded89ff2-fe00-429c-9161-673297bcab80 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:31.592867 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340493a-8102-465c-bda8-30f59b54947a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8340493a-8102-465c-bda8-30f59b54947a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:31.598049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:31.599260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:31.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:31.604207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 529/1056] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 31 12:06:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:31.605301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340493a-8102-465c-bda8-30f59b54947a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:31.651430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7be54-0d10-4a83-8633-f262d2432010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:31.652510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7be54-0d10-4a83-8633-f262d2432010 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:31.652788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7be54-0d10-4a83-8633-f262d2432010 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:31.653093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7be54-0d10-4a83-8633-f262d2432010 None admin] PATH_INFO: `/v3/projects/92013b606d324e169a80498cfdc68734` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:31.654262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7be54-0d10-4a83-8633-f262d2432010 None admin] RBAC: Authorizing `identity:get_project(project_id=92013b606d324e169a80498cfdc68734)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:31.655986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7be54-0d10-4a83-8633-f262d2432010 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:31.659567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 528/1057] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:06:31 2022] GET /identity/v3/projects/92013b606d324e169a80498cfdc68734 => generated 311 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:31.664529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7be54-0d10-4a83-8633-f262d2432010 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:31.717495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340493a-8102-465c-bda8-30f59b54947a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:31.724612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:31.762817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:31.763802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:31.764249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:31.764690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:31.765418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:31.765875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:31.768242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:31.781254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 530/1058] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 31 12:06:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:31.782560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:31.809987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7be54-0d10-4a83-8633-f262d2432010 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:31.816236 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:31.852676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:31.853285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:31.853558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:31.853826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:31.854713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=92013b606d324e169a80498cfdc68734, user.id=f6a74e43498847089af815b4cd6bfd1a) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:31.855456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:31.856646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:31.875485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 529/1059] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:31 2022] GET /identity/v3/role_assignments?user.id=f6a74e43498847089af815b4cd6bfd1a&scope.project.id=92013b606d324e169a80498cfdc68734&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:31.875485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:31.908450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2545-15f9-42d6-b29c-b9536dba3125 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:31.947122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:31.948060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:31.948409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:31.948944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] PATH_INFO: `/v3/services/image` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:31.949774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:31.951657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:31.961807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:31.962885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:31.963824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 531/1060] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 31 12:06:31 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:31.969632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:32.002137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ac4a3-2b9e-42a0-a6c5-06b71fee8343 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:32.004987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:32.056193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:32.057126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:32.059110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:32.059421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:32.074752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:32.076275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:32.080837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:32.093553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:32.094754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:32.097488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:32.097488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:32.097488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:32.097488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 530/1061] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:32.101675 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:32.106574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c5be3-4fdc-42b1-b3fb-45550efbfe3b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:32.153621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:32.154240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:32.154519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:32.154789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:32.155466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:32.155765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:32.157176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:32.162748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 532/1062] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:32.169010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:32.224559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7124cb-328e-4eb7-b9cd-560f79e2b855 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:32.277396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:32.278143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:32.278489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:32.278831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:32.279553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:32.279871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:32.281266 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:32.295623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 531/1063] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:32.301391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:32.319453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c0eec-89de-4329-a17a-e650c57bc0a7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:32.325690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:32.372196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:32.372722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:32.372999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:32.373270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:32.373911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:32.374257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:32.375743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:32.382180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 533/1064] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:32.390447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:32.418140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:32.443565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5881b7-5490-45d8-b234-9c82fafda986 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:32.493958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:32.494524 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:32.494797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:32.495117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:32.503980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:32.505442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:06:32.517832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:32.518853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:32.519867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 532/1065] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:06:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:32.524657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:32.532967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4761a-59e3-434c-a8bf-2e43e8e019f5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:32.570577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3bcc0-5ca2-43af-88c2-6429423566bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:32.571195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3bcc0-5ca2-43af-88c2-6429423566bd None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:32.571623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3bcc0-5ca2-43af-88c2-6429423566bd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:32.571825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3bcc0-5ca2-43af-88c2-6429423566bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:32.572485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3bcc0-5ca2-43af-88c2-6429423566bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:32.574112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3bcc0-5ca2-43af-88c2-6429423566bd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:32.595861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 534/1066] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:06:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:06:32.621480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:32.651571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0c6cd-c9bd-49cb-98e5-235fc0042af6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:32.715538 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:32.717009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:32.717736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:32.718446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:32.720200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:32.720988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:32.724429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:32.757753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 533/1067] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:06:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:32.764224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3bcc0-5ca2-43af-88c2-6429423566bd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:32.863903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:32.918459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3bcc0-5ca2-43af-88c2-6429423566bd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:32.966873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:32.968083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:32.968659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:32.969189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:32.988446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:32.991066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:33.016240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:33.017404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:33.018486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:33.018486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:33.018486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:33.020616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 535/1068] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:33.029406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:33.176419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbaec3-a15f-47ce-ae8f-6ac0e1b793d8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:33.208732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:33.216651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:33.217289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:33.217569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:33.217837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:33.218503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:33.218804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:33.220259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:33.233721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 534/1069] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:33.239374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:33.381292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f9b6f-7dfb-4dc0-a8af-a7189c74b7e0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:33.413114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:33.429933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:33.430563 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:33.430822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:33.431081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:33.438618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:33.440085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:33.451752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:33.452699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:33.453672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 536/1070] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:33.460211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:33.568354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22e18b-1d3b-4365-a011-c26821db440d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:33.605745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d8e17-6a8b-435e-bcc4-f53b47b9e62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:33.606376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d8e17-6a8b-435e-bcc4-f53b47b9e62d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:33.606703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d8e17-6a8b-435e-bcc4-f53b47b9e62d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:33.606894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d8e17-6a8b-435e-bcc4-f53b47b9e62d None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:33.607656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d8e17-6a8b-435e-bcc4-f53b47b9e62d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:33.607958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d8e17-6a8b-435e-bcc4-f53b47b9e62d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:33.609396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d8e17-6a8b-435e-bcc4-f53b47b9e62d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:33.627347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 535/1071] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:33.630009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:33.746277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673bf53-b4a3-4884-a91b-78bbd0b745bd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:33.776006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2f36f-d24e-444a-86f7-727148c1e34e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:33.776451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2f36f-d24e-444a-86f7-727148c1e34e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:33.776661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2f36f-d24e-444a-86f7-727148c1e34e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:33.778014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 536/1072] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:06:33.783094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adfb37-aad8-4d5c-9738-6c8d28535c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:33.783650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23290c-0393-4239-9719-c89326749c63 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:33.783650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23290c-0393-4239-9719-c89326749c63 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:33.783650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23290c-0393-4239-9719-c89326749c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:33.783960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adfb37-aad8-4d5c-9738-6c8d28535c89 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:33.784332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adfb37-aad8-4d5c-9738-6c8d28535c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:33.784648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adfb37-aad8-4d5c-9738-6c8d28535c89 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/0b113316bc3348b5a12c1d7b616bf4c9/roles/0b69eefec9d249b39b3ef87c6de99c50` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:33.786876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adfb37-aad8-4d5c-9738-6c8d28535c89 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=0b113316bc3348b5a12c1d7b616bf4c9, role_id=0b69eefec9d249b39b3ef87c6de99c50)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:33.788408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adfb37-aad8-4d5c-9738-6c8d28535c89 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:33.802571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6d23290c-0393-4239-9719-c89326749c63 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:33.804124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6d23290c-0393-4239-9719-c89326749c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:33.819375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 537/1073] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:06:33 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/0b113316bc3348b5a12c1d7b616bf4c9/roles/0b69eefec9d249b39b3ef87c6de99c50 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:06:33.860619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:33.887918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 537/1074] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:34.159794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adfb37-aad8-4d5c-9738-6c8d28535c89 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:34.205548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:34.271285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adfb37-aad8-4d5c-9738-6c8d28535c89 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:34.305464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c8a0f-0be3-43b1-9a1c-ef2a1469eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:34.306250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8a0f-0be3-43b1-9a1c-ef2a1469eb06 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:34.306614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8a0f-0be3-43b1-9a1c-ef2a1469eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:34.306972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8a0f-0be3-43b1-9a1c-ef2a1469eb06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:34.317495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c8a0f-0be3-43b1-9a1c-ef2a1469eb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-306c8a0f-0be3-43b1-9a1c-ef2a1469eb06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:34.322899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:34.324561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:06:34.325626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:34.327028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 538/1075] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:06:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:34.333036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23290c-0393-4239-9719-c89326749c63 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:34.462652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23290c-0393-4239-9719-c89326749c63 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:34.500154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9ec19-3ac6-462d-9373-795fae1b108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:34.500766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ec19-3ac6-462d-9373-795fae1b108d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:34.501037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ec19-3ac6-462d-9373-795fae1b108d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:34.501298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ec19-3ac6-462d-9373-795fae1b108d None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:34.501950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9ec19-3ac6-462d-9373-795fae1b108d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:34.502246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9ec19-3ac6-462d-9373-795fae1b108d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:34.503671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9ec19-3ac6-462d-9373-795fae1b108d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:34.512715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 538/1076] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:06:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:34.518385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c8a0f-0be3-43b1-9a1c-ef2a1469eb06 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:34.544940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759e6ea-1d50-406e-ac19-65abe413508d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:34.545253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759e6ea-1d50-406e-ac19-65abe413508d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:34.545533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759e6ea-1d50-406e-ac19-65abe413508d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:34.546964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 539/1077] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:34.552958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983baf8-0bf7-43b5-bf77-2a5dd9c6efba None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:34.553294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983baf8-0bf7-43b5-bf77-2a5dd9c6efba None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:34.553587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983baf8-0bf7-43b5-bf77-2a5dd9c6efba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:34.572953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9983baf8-0bf7-43b5-bf77-2a5dd9c6efba None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:34.574250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9983baf8-0bf7-43b5-bf77-2a5dd9c6efba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:34.615295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:34.627255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:34.631355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c8a0f-0be3-43b1-9a1c-ef2a1469eb06 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:34.663587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 540/1078] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:34.670921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c2b87-a557-411d-8bfc-d789e5c18c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:34.671680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c2b87-a557-411d-8bfc-d789e5c18c47 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:34.672020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c2b87-a557-411d-8bfc-d789e5c18c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:34.672350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c2b87-a557-411d-8bfc-d789e5c18c47 None admin] PATH_INFO: `/v3/users/f6a74e43498847089af815b4cd6bfd1a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:34.673550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c2b87-a557-411d-8bfc-d789e5c18c47 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a74e43498847089af815b4cd6bfd1a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:34.675060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c2b87-a557-411d-8bfc-d789e5c18c47 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:34.676935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 539/1079] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:34 2022] GET /identity/v3/users/f6a74e43498847089af815b4cd6bfd1a => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:34.682635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983baf8-0bf7-43b5-bf77-2a5dd9c6efba None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:34.788257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983baf8-0bf7-43b5-bf77-2a5dd9c6efba None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:34.825955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137181ef-7c56-4110-88d3-36ee693b850a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:34.826591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137181ef-7c56-4110-88d3-36ee693b850a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:34.826859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137181ef-7c56-4110-88d3-36ee693b850a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:34.827158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137181ef-7c56-4110-88d3-36ee693b850a None admin] PATH_INFO: `/v3/projects/92013b606d324e169a80498cfdc68734` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:34.828290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137181ef-7c56-4110-88d3-36ee693b850a None admin] RBAC: Authorizing `identity:get_project(project_id=92013b606d324e169a80498cfdc68734)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:34.829708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137181ef-7c56-4110-88d3-36ee693b850a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:34.831561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 541/1080] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:06:34 2022] GET /identity/v3/projects/92013b606d324e169a80498cfdc68734 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:34.837325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c2b87-a557-411d-8bfc-d789e5c18c47 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:34.916120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137181ef-7c56-4110-88d3-36ee693b850a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:34.963457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c2b87-a557-411d-8bfc-d789e5c18c47 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:34.973648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:35.004279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31faee-e2f1-442b-b0eb-b02869c66093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:35.004952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31faee-e2f1-442b-b0eb-b02869c66093 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:35.005233 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31faee-e2f1-442b-b0eb-b02869c66093 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:35.005509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31faee-e2f1-442b-b0eb-b02869c66093 None admin] PATH_INFO: `/v3/projects/92013b606d324e169a80498cfdc68734/users/f6a74e43498847089af815b4cd6bfd1a/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:35.007755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31faee-e2f1-442b-b0eb-b02869c66093 None admin] RBAC: Authorizing `identity:create_grant(project_id=92013b606d324e169a80498cfdc68734, user_id=f6a74e43498847089af815b4cd6bfd1a, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:35.009204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31faee-e2f1-442b-b0eb-b02869c66093 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:35.026372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 540/1081] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:06:34 2022] PUT /identity/v3/projects/92013b606d324e169a80498cfdc68734/users/f6a74e43498847089af815b4cd6bfd1a/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:06:35.049472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137181ef-7c56-4110-88d3-36ee693b850a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:35.088719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ebd4d-15eb-4118-b79f-b151dc6c08da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:35.089547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ebd4d-15eb-4118-b79f-b151dc6c08da None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:35.089791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ebd4d-15eb-4118-b79f-b151dc6c08da None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:35.090091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ebd4d-15eb-4118-b79f-b151dc6c08da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:35.177198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ebd4d-15eb-4118-b79f-b151dc6c08da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:35.178681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ebd4d-15eb-4118-b79f-b151dc6c08da None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:35.224575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:35.291269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 542/1082] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 12:06:35.294037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31faee-e2f1-442b-b0eb-b02869c66093 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:35.326573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:35.410269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31faee-e2f1-442b-b0eb-b02869c66093 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:35.448395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3cf20-30d0-4af7-8faf-adf18198fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:35.449557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3cf20-30d0-4af7-8faf-adf18198fa37 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:35.449838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3cf20-30d0-4af7-8faf-adf18198fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:35.450106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3cf20-30d0-4af7-8faf-adf18198fa37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:35.463030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3cf20-30d0-4af7-8faf-adf18198fa37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:35.465361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3cf20-30d0-4af7-8faf-adf18198fa37 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-02e3cf20-30d0-4af7-8faf-adf18198fa37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:35.481331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:35.482352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:35.483313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:35.483313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:35.483313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:35.487324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 541/1083] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:35.488845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ebd4d-15eb-4118-b79f-b151dc6c08da None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:35.599134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ebd4d-15eb-4118-b79f-b151dc6c08da None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:35.635931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3356b7-4297-466e-a5c5-51a2eb8dca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:35.636692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3356b7-4297-466e-a5c5-51a2eb8dca60 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:35.636971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3356b7-4297-466e-a5c5-51a2eb8dca60 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:35.637243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3356b7-4297-466e-a5c5-51a2eb8dca60 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:35.637900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3356b7-4297-466e-a5c5-51a2eb8dca60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:35.638201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3356b7-4297-466e-a5c5-51a2eb8dca60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:35.639619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3356b7-4297-466e-a5c5-51a2eb8dca60 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:35.648749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:35.657500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 543/1084] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:35.734421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6824615-6f95-4608-9940-3d897d2802a6 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:35.734770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6824615-6f95-4608-9940-3d897d2802a6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:35.734970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6824615-6f95-4608-9940-3d897d2802a6 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:35.736645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 542/1085] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:35.743833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3bddb-6347-45d7-857f-c937f20cfef9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:35.744144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3bddb-6347-45d7-857f-c937f20cfef9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:35.744403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3bddb-6347-45d7-857f-c937f20cfef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:35.767542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7ce3bddb-6347-45d7-857f-c937f20cfef9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:35.768824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7ce3bddb-6347-45d7-857f-c937f20cfef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:35.857948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 544/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:36.131408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6824615-6f95-4608-9940-3d897d2802a6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:36.227817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:36.250368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6824615-6f95-4608-9940-3d897d2802a6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:36.289219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42b3ad-a321-40f3-8159-cffb53e6fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:36.289870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b3ad-a321-40f3-8159-cffb53e6fb8c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:36.290145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b3ad-a321-40f3-8159-cffb53e6fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:36.290414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b3ad-a321-40f3-8159-cffb53e6fb8c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:36.301361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42b3ad-a321-40f3-8159-cffb53e6fb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9b42b3ad-a321-40f3-8159-cffb53e6fb8c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:36.306080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:36.307063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:36.308485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 543/1087] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 31 12:06:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:36.313626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3bddb-6347-45d7-857f-c937f20cfef9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:36.421887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3bddb-6347-45d7-857f-c937f20cfef9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:36.452795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:36.459680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:36.460300 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:36.460573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:36.460843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:36.461517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:36.461824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:36.463256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:36.472782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 545/1088] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 31 12:06:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:36.478687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42b3ad-a321-40f3-8159-cffb53e6fb8c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:36.590701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42b3ad-a321-40f3-8159-cffb53e6fb8c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:36.627786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:36.628435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:36.628710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:36.628950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:36.639032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:36.641234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:36.642894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:36.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:36.660452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:36.661362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:36.661362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:36.661362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:36.661362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 544/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:36.666621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:36.778640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46ffa1-e003-4b36-8ed0-c7796672091a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:36.815125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:36.816141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:36.816396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:36.816687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:36.817329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:36.817630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:36.819015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:36.832234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 546/1090] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:36.838111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:36.950289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98c12d-6347-48d8-a4eb-a0d5c011a8e9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:36.987114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a210f8-ed8d-492a-9ca3-b0f99dc3deb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:36.987782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a210f8-ed8d-492a-9ca3-b0f99dc3deb4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:36.988057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a210f8-ed8d-492a-9ca3-b0f99dc3deb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:36.988327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a210f8-ed8d-492a-9ca3-b0f99dc3deb4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:36.997397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a210f8-ed8d-492a-9ca3-b0f99dc3deb4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:36.998849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a210f8-ed8d-492a-9ca3-b0f99dc3deb4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-57a210f8-ed8d-492a-9ca3-b0f99dc3deb4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:06:37.011273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:37.012568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:37.013535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 545/1091] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 31 12:06:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:37.018594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.039343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff500e1-06b3-4387-8e03-49166e669353 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.039343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff500e1-06b3-4387-8e03-49166e669353 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.039645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff500e1-06b3-4387-8e03-49166e669353 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.040849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 546/1092] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:37.046426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde455-24b7-4e96-b1b9-fbc34758d8d0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.046688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde455-24b7-4e96-b1b9-fbc34758d8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.046957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde455-24b7-4e96-b1b9-fbc34758d8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.066833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9bbde455-24b7-4e96-b1b9-fbc34758d8d0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:37.068081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9bbde455-24b7-4e96-b1b9-fbc34758d8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:37.075507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:37.118903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:37.127265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae48f3-e3f5-4522-bf52-3a45ab035501 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:37.140963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 547/1093] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:37.167959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:37.168690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.168968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.169232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.169970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:37.170271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:37.171773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:37.199608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 547/1094] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:37.205557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbde455-24b7-4e96-b1b9-fbc34758d8d0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.307998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbde455-24b7-4e96-b1b9-fbc34758d8d0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:37.342605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aff30-6d77-45ef-acaa-4d171ca81c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:37.343220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aff30-6d77-45ef-acaa-4d171ca81c64 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.343522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aff30-6d77-45ef-acaa-4d171ca81c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.343794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aff30-6d77-45ef-acaa-4d171ca81c64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.398283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:37.410462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.421970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772aff30-6d77-45ef-acaa-4d171ca81c64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:37.423619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772aff30-6d77-45ef-acaa-4d171ca81c64 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:37.516411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6be3-590d-40b5-b5fb-204c947c00e5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:37.527977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:37.529842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 548/1095] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:06:37.535404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aff30-6d77-45ef-acaa-4d171ca81c64 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.554403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd7789-f564-45cb-adfd-a22ab3dfe9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:37.555030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd7789-f564-45cb-adfd-a22ab3dfe9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.555393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd7789-f564-45cb-adfd-a22ab3dfe9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.555788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd7789-f564-45cb-adfd-a22ab3dfe9e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.567184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd7789-f564-45cb-adfd-a22ab3dfe9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b7dd7789-f564-45cb-adfd-a22ab3dfe9e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:37.571807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:37.572836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:37.573696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 548/1096] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:37.582054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd7789-f564-45cb-adfd-a22ab3dfe9e6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.627317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:37.646192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aff30-6d77-45ef-acaa-4d171ca81c64 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:37.681761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:37.682816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.683089 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.683393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.689820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd7789-f564-45cb-adfd-a22ab3dfe9e6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:37.694375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:37.695868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:06:37.710858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:37.712032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:06:37.712938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:06:37.712938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:06:37.712938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:37.712938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 549/1097] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:37.718800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.729245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:37.729941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.730222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.730501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.731180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:37.731514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:37.732915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:37.742239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 549/1098] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:37.748221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.780588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:37.830346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd7145-26a3-4dd9-8ab7-fb08b56f67e8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:37.856419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0309b6-1437-4919-8591-8dfdeb966e0a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:37.866224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:37.866846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.867129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.867434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.868079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:37.868401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:37.869804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:37.871516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:37.884446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 550/1099] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:37.890688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.900622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eeb8f-667a-4240-82dd-685cbcfd758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:37.901435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eeb8f-667a-4240-82dd-685cbcfd758a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:37.901834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eeb8f-667a-4240-82dd-685cbcfd758a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:37.902117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eeb8f-667a-4240-82dd-685cbcfd758a None admin] PATH_INFO: `/v3/users/f6a74e43498847089af815b4cd6bfd1a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:37.903419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eeb8f-667a-4240-82dd-685cbcfd758a None admin] RBAC: Authorizing `identity:get_user(user_id=f6a74e43498847089af815b4cd6bfd1a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:37.904903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eeb8f-667a-4240-82dd-685cbcfd758a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:37.906792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 550/1100] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/users/f6a74e43498847089af815b4cd6bfd1a => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:37.912726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eeb8f-667a-4240-82dd-685cbcfd758a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:37.992671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f25b9-f5a8-401a-bdff-12c296263c86 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:38.010697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:38.014455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eeb8f-667a-4240-82dd-685cbcfd758a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:38.029679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:38.030305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:38.030584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:38.030853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:38.037673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:38.039114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:06:38.050894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:38.051897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:38.052790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:06:38.052790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:06:38.052790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:06:38.052790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:06:38.052790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:06:38.052790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:06:38.052790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:06:38.052790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:06:38.053230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426ffe1-4f06-4bf7-b68b-7f80fafd5648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:38.053323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 551/1101] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:06:38.053402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426ffe1-4f06-4bf7-b68b-7f80fafd5648 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:38.053402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426ffe1-4f06-4bf7-b68b-7f80fafd5648 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:38.053402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426ffe1-4f06-4bf7-b68b-7f80fafd5648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:38.058537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:38.114640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:38.125741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426ffe1-4f06-4bf7-b68b-7f80fafd5648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:38.127260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426ffe1-4f06-4bf7-b68b-7f80fafd5648 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:38.161798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df80d72-a077-489e-95da-87db358dcbb1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:38.202350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:38.202965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:38.203265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:38.203543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:38.204244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:38.204547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:38.205995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:38.218736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 552/1102] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:38.225343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:38.254178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:38.263988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 551/1103] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 12:06:38.265202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426ffe1-4f06-4bf7-b68b-7f80fafd5648 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:38.332599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77d70a-e813-4c2d-8b90-a74098b323be None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:38.347759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:38.369180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426ffe1-4f06-4bf7-b68b-7f80fafd5648 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:38.370025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:38.370697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:38.371101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:38.371413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:38.372473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] RBAC: Adding query filter params (role.id=0b69eefec9d249b39b3ef87c6de99c50, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=0b113316bc3348b5a12c1d7b616bf4c9) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:38.372788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:38.374237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:38.380576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 553/1104] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:38 2022] GET /identity/v3/role_assignments?user.id=0b113316bc3348b5a12c1d7b616bf4c9&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=0b69eefec9d249b39b3ef87c6de99c50 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:38.409604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afe79b-d0d5-4b08-83c9-5b2ddcc1badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:38.410259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe79b-d0d5-4b08-83c9-5b2ddcc1badc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:38.410622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe79b-d0d5-4b08-83c9-5b2ddcc1badc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:38.410817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe79b-d0d5-4b08-83c9-5b2ddcc1badc None admin] PATH_INFO: `/v3/projects/92013b606d324e169a80498cfdc68734` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:38.412072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afe79b-d0d5-4b08-83c9-5b2ddcc1badc None admin] RBAC: Authorizing `identity:get_project(project_id=92013b606d324e169a80498cfdc68734)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:38.413559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afe79b-d0d5-4b08-83c9-5b2ddcc1badc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:38.415348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 552/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:06:38 2022] GET /identity/v3/projects/92013b606d324e169a80498cfdc68734 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:06:38.421719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:38.547459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db0c0a-cd34-4572-863e-dfc420344172 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:38.608880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45a569-e726-4d8d-ac18-8a60c6153b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:38.609502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a569-e726-4d8d-ac18-8a60c6153b90 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:38.609774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a569-e726-4d8d-ac18-8a60c6153b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:38.610039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a569-e726-4d8d-ac18-8a60c6153b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:38.611023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45a569-e726-4d8d-ac18-8a60c6153b90 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=92013b606d324e169a80498cfdc68734, user.id=f6a74e43498847089af815b4cd6bfd1a) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:38.611535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45a569-e726-4d8d-ac18-8a60c6153b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:38.613804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45a569-e726-4d8d-ac18-8a60c6153b90 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:38.622184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 554/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:38 2022] GET /identity/v3/role_assignments?user.id=f6a74e43498847089af815b4cd6bfd1a&scope.project.id=92013b606d324e169a80498cfdc68734&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:40.631553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0618d4-f3bc-4dfb-a657-5094bfb08eca None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:40.632207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0618d4-f3bc-4dfb-a657-5094bfb08eca None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:40.632207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0618d4-f3bc-4dfb-a657-5094bfb08eca None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:40.633516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 553/1107] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:40.639686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b808d0-c76a-43e1-92c8-cdb16a357eef None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:40.640392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b808d0-c76a-43e1-92c8-cdb16a357eef None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:40.640392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b808d0-c76a-43e1-92c8-cdb16a357eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:40.661027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-36b808d0-c76a-43e1-92c8-cdb16a357eef None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:40.662260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-36b808d0-c76a-43e1-92c8-cdb16a357eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:40.701941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:40.744555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 555/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:06:41.008742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0618d4-f3bc-4dfb-a657-5094bfb08eca None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:41.085022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:41.125862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0618d4-f3bc-4dfb-a657-5094bfb08eca None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:41.172685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff21fa-84ce-4705-a5e2-ef39883b7209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:41.173559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff21fa-84ce-4705-a5e2-ef39883b7209 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:41.173839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff21fa-84ce-4705-a5e2-ef39883b7209 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:41.174110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff21fa-84ce-4705-a5e2-ef39883b7209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:41.175493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff21fa-84ce-4705-a5e2-ef39883b7209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:41.177158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff21fa-84ce-4705-a5e2-ef39883b7209 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:41.179050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 554/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:41.185847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b808d0-c76a-43e1-92c8-cdb16a357eef None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:41.298915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b808d0-c76a-43e1-92c8-cdb16a357eef None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:41.335504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b343612-a293-49d7-888b-3d04c767d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:41.336117 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b343612-a293-49d7-888b-3d04c767d602 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:41.336388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b343612-a293-49d7-888b-3d04c767d602 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:41.336655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b343612-a293-49d7-888b-3d04c767d602 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:41.337329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b343612-a293-49d7-888b-3d04c767d602 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:41.339603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b343612-a293-49d7-888b-3d04c767d602 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:41.363186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 556/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:06:41 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:06:43.320153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aaa72-e540-4287-b00f-97cd241cd03c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:43.320770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aaa72-e540-4287-b00f-97cd241cd03c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:43.320770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aaa72-e540-4287-b00f-97cd241cd03c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:43.322043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 555/1111] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:43.327675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231677af-59c9-4aac-a5f5-a70dd1b26c77 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:43.328227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231677af-59c9-4aac-a5f5-a70dd1b26c77 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:43.328227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231677af-59c9-4aac-a5f5-a70dd1b26c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:43.348131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-231677af-59c9-4aac-a5f5-a70dd1b26c77 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:43.349371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-231677af-59c9-4aac-a5f5-a70dd1b26c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:43.377364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:43.427949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 557/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:43.693891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aaa72-e540-4287-b00f-97cd241cd03c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:43.806113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aaa72-e540-4287-b00f-97cd241cd03c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:43.859630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacc301-0d36-4100-93ca-e511c5918763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:43.860395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacc301-0d36-4100-93ca-e511c5918763 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:43.860859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacc301-0d36-4100-93ca-e511c5918763 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:43.861038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacc301-0d36-4100-93ca-e511c5918763 None admin] PATH_INFO: `/v3/roles/0f4eb127111b4fdabec35268db5bde9e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:43.862626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacc301-0d36-4100-93ca-e511c5918763 None admin] RBAC: Authorizing `identity:get_role(role_id=0f4eb127111b4fdabec35268db5bde9e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:43.864153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacc301-0d36-4100-93ca-e511c5918763 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:43.865601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 556/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:43 2022] GET /identity/v3/roles/0f4eb127111b4fdabec35268db5bde9e => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:43.874630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231677af-59c9-4aac-a5f5-a70dd1b26c77 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:43.984783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231677af-59c9-4aac-a5f5-a70dd1b26c77 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:44.021684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a763e-1b2b-4dc3-bc7f-051b3320a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:44.022336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a763e-1b2b-4dc3-bc7f-051b3320a69b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:44.022625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a763e-1b2b-4dc3-bc7f-051b3320a69b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:44.022864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a763e-1b2b-4dc3-bc7f-051b3320a69b None admin] PATH_INFO: `/v3/users/6ff1253ebd3d4ccb967c8c9b1c209d8b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:44.038329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a763e-1b2b-4dc3-bc7f-051b3320a69b None admin] RBAC: Authorizing `identity:get_user(user_id=6ff1253ebd3d4ccb967c8c9b1c209d8b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:44.039804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a763e-1b2b-4dc3-bc7f-051b3320a69b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:44.041562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 558/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:43 2022] GET /identity/v3/users/6ff1253ebd3d4ccb967c8c9b1c209d8b => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:44.047758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacc301-0d36-4100-93ca-e511c5918763 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:44.052174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:44.156943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacc301-0d36-4100-93ca-e511c5918763 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:44.195063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0ce0c-c6f1-4510-8028-f3cd9fb43a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:44.195704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0ce0c-c6f1-4510-8028-f3cd9fb43a7d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:44.195981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0ce0c-c6f1-4510-8028-f3cd9fb43a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:44.196252 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0ce0c-c6f1-4510-8028-f3cd9fb43a7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:44.274283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0ce0c-c6f1-4510-8028-f3cd9fb43a7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:44.275736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0ce0c-c6f1-4510-8028-f3cd9fb43a7d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:44.279264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:44.391106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 557/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:06:44.397481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a763e-1b2b-4dc3-bc7f-051b3320a69b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:44.416336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:44.504787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a763e-1b2b-4dc3-bc7f-051b3320a69b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:44.539214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd17ff-2b3d-41e5-9810-097bb37096e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:44.539806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd17ff-2b3d-41e5-9810-097bb37096e9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:44.540081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd17ff-2b3d-41e5-9810-097bb37096e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:44.540346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd17ff-2b3d-41e5-9810-097bb37096e9 None admin] PATH_INFO: `/v3/projects/92013b606d324e169a80498cfdc68734` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:44.541751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd17ff-2b3d-41e5-9810-097bb37096e9 None admin] RBAC: Authorizing `identity:get_project(project_id=92013b606d324e169a80498cfdc68734)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:44.542974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd17ff-2b3d-41e5-9810-097bb37096e9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:44.544652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 559/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:06:44 2022] GET /identity/v3/projects/92013b606d324e169a80498cfdc68734 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:44.551171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0ce0c-c6f1-4510-8028-f3cd9fb43a7d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:44.658814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0ce0c-c6f1-4510-8028-f3cd9fb43a7d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:44.661247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:44.695853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d7879-5114-47a7-a342-5af9254246bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:44.696456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d7879-5114-47a7-a342-5af9254246bc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:44.696730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d7879-5114-47a7-a342-5af9254246bc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:44.696998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d7879-5114-47a7-a342-5af9254246bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:44.697817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d7879-5114-47a7-a342-5af9254246bc None admin] RBAC: Adding query filter params (role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=92013b606d324e169a80498cfdc68734, user.id=6ff1253ebd3d4ccb967c8c9b1c209d8b) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:44.698603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d7879-5114-47a7-a342-5af9254246bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:44.700072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d7879-5114-47a7-a342-5af9254246bc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:44.705768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 558/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:44 2022] GET /identity/v3/role_assignments?user.id=6ff1253ebd3d4ccb967c8c9b1c209d8b&scope.project.id=92013b606d324e169a80498cfdc68734&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:46.614050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9d50-44c7-4676-a516-cd8dabe5ca53 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:46.614651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9d50-44c7-4676-a516-cd8dabe5ca53 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:46.614651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9d50-44c7-4676-a516-cd8dabe5ca53 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:46.615977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 560/1118] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:46.621513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf880ba2-098c-4f66-bbeb-33bf796a2a1a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:46.621974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf880ba2-098c-4f66-bbeb-33bf796a2a1a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:46.622071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf880ba2-098c-4f66-bbeb-33bf796a2a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:46.641250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bf880ba2-098c-4f66-bbeb-33bf796a2a1a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:46.642471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bf880ba2-098c-4f66-bbeb-33bf796a2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:46.718126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 559/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:46.986479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9d50-44c7-4676-a516-cd8dabe5ca53 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:47.076046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:47.094098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9d50-44c7-4676-a516-cd8dabe5ca53 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:47.132609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846362a7-1901-41e2-8074-aa0d11d0112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:47.133257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846362a7-1901-41e2-8074-aa0d11d0112f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:47.133533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846362a7-1901-41e2-8074-aa0d11d0112f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:47.133804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846362a7-1901-41e2-8074-aa0d11d0112f None admin] PATH_INFO: `/v3/roles/0f4eb127111b4fdabec35268db5bde9e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:47.135569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846362a7-1901-41e2-8074-aa0d11d0112f None admin] RBAC: Authorizing `identity:get_role(role_id=0f4eb127111b4fdabec35268db5bde9e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:47.137234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846362a7-1901-41e2-8074-aa0d11d0112f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:47.138632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 561/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:46 2022] GET /identity/v3/roles/0f4eb127111b4fdabec35268db5bde9e => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:47.144802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf880ba2-098c-4f66-bbeb-33bf796a2a1a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:47.249764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:47.253732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf880ba2-098c-4f66-bbeb-33bf796a2a1a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:47.292913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d89bb7-d1a5-4845-8dc8-898032718c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:47.293588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d89bb7-d1a5-4845-8dc8-898032718c45 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:47.293865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d89bb7-d1a5-4845-8dc8-898032718c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:47.294142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d89bb7-d1a5-4845-8dc8-898032718c45 None admin] PATH_INFO: `/v3/users/6ff1253ebd3d4ccb967c8c9b1c209d8b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:47.295365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d89bb7-d1a5-4845-8dc8-898032718c45 None admin] RBAC: Authorizing `identity:get_user(user_id=6ff1253ebd3d4ccb967c8c9b1c209d8b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:47.296826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d89bb7-d1a5-4845-8dc8-898032718c45 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:47.298483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 560/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:47 2022] GET /identity/v3/users/6ff1253ebd3d4ccb967c8c9b1c209d8b => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:47.304600 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846362a7-1901-41e2-8074-aa0d11d0112f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:47.414661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846362a7-1901-41e2-8074-aa0d11d0112f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:47.451570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249b241-d548-4798-96ab-b35ccd2ed606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:47.452229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249b241-d548-4798-96ab-b35ccd2ed606 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:47.452500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249b241-d548-4798-96ab-b35ccd2ed606 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:47.452851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249b241-d548-4798-96ab-b35ccd2ed606 None admin] PATH_INFO: `/v3/projects/92013b606d324e169a80498cfdc68734` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:47.454086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249b241-d548-4798-96ab-b35ccd2ed606 None admin] RBAC: Authorizing `identity:get_project(project_id=92013b606d324e169a80498cfdc68734)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:47.455579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249b241-d548-4798-96ab-b35ccd2ed606 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:47.457226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 562/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:06:47 2022] GET /identity/v3/projects/92013b606d324e169a80498cfdc68734 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:47.463784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d89bb7-d1a5-4845-8dc8-898032718c45 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:47.574404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d89bb7-d1a5-4845-8dc8-898032718c45 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:47.610843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81635b0-db21-40df-af6e-d2710da7b995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:47.611510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81635b0-db21-40df-af6e-d2710da7b995 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:47.611780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81635b0-db21-40df-af6e-d2710da7b995 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:47.612048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81635b0-db21-40df-af6e-d2710da7b995 None admin] PATH_INFO: `/v3/projects/92013b606d324e169a80498cfdc68734/users/6ff1253ebd3d4ccb967c8c9b1c209d8b/roles/0f4eb127111b4fdabec35268db5bde9e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:47.614214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81635b0-db21-40df-af6e-d2710da7b995 None admin] RBAC: Authorizing `identity:create_grant(project_id=92013b606d324e169a80498cfdc68734, user_id=6ff1253ebd3d4ccb967c8c9b1c209d8b, role_id=0f4eb127111b4fdabec35268db5bde9e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:47.615680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81635b0-db21-40df-af6e-d2710da7b995 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:47.629884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 561/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:06:47 2022] PUT /identity/v3/projects/92013b606d324e169a80498cfdc68734/users/6ff1253ebd3d4ccb967c8c9b1c209d8b/roles/0f4eb127111b4fdabec35268db5bde9e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:06:49.560336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cdb02-dd05-4189-8674-de447b1a6809 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:49.560925 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cdb02-dd05-4189-8674-de447b1a6809 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:49.560925 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cdb02-dd05-4189-8674-de447b1a6809 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:49.562228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 563/1124] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:49.568154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916031d-7202-45a0-baca-4d6d882c9e7a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:49.568638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916031d-7202-45a0-baca-4d6d882c9e7a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:49.568754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916031d-7202-45a0-baca-4d6d882c9e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:49.588608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0916031d-7202-45a0-baca-4d6d882c9e7a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:49.590359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0916031d-7202-45a0-baca-4d6d882c9e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:49.613635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:49.667115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 562/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:49.931695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cdb02-dd05-4189-8674-de447b1a6809 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:49.954614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:50.047149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cdb02-dd05-4189-8674-de447b1a6809 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:50.090697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd99447-de8f-4276-b70f-65da6fadba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:50.091348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99447-de8f-4276-b70f-65da6fadba44 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:50.091624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99447-de8f-4276-b70f-65da6fadba44 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:50.091894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99447-de8f-4276-b70f-65da6fadba44 None admin] PATH_INFO: `/v3/roles/0f4eb127111b4fdabec35268db5bde9e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:50.093336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd99447-de8f-4276-b70f-65da6fadba44 None admin] RBAC: Authorizing `identity:get_role(role_id=0f4eb127111b4fdabec35268db5bde9e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:50.094710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd99447-de8f-4276-b70f-65da6fadba44 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:50.096065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 564/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:49 2022] GET /identity/v3/roles/0f4eb127111b4fdabec35268db5bde9e => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:50.102564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916031d-7202-45a0-baca-4d6d882c9e7a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:50.208240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916031d-7202-45a0-baca-4d6d882c9e7a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:50.248043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f546-1d20-42ae-bcec-180fae830274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:50.248709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f546-1d20-42ae-bcec-180fae830274 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:50.248984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f546-1d20-42ae-bcec-180fae830274 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:50.249249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f546-1d20-42ae-bcec-180fae830274 None admin] PATH_INFO: `/v3/users/6ff1253ebd3d4ccb967c8c9b1c209d8b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:50.250529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0f546-1d20-42ae-bcec-180fae830274 None admin] RBAC: Authorizing `identity:get_user(user_id=6ff1253ebd3d4ccb967c8c9b1c209d8b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:50.252037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0f546-1d20-42ae-bcec-180fae830274 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:50.253729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 563/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:50 2022] GET /identity/v3/users/6ff1253ebd3d4ccb967c8c9b1c209d8b => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:50.260173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd99447-de8f-4276-b70f-65da6fadba44 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:50.351050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:50.363970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd99447-de8f-4276-b70f-65da6fadba44 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:50.399071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f72a7a-c687-4ddc-9ee9-b21a78ef26e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:50.399708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72a7a-c687-4ddc-9ee9-b21a78ef26e7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:50.399980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72a7a-c687-4ddc-9ee9-b21a78ef26e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:50.400246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72a7a-c687-4ddc-9ee9-b21a78ef26e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:50.468949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f72a7a-c687-4ddc-9ee9-b21a78ef26e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:50.470377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f72a7a-c687-4ddc-9ee9-b21a78ef26e7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:50.558063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:50.571758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 565/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:06:50.578687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f546-1d20-42ae-bcec-180fae830274 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:50.613285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:50.695162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f546-1d20-42ae-bcec-180fae830274 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:50.731882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7182-ef13-444d-b6a7-ee3c7df42c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:50.732425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7182-ef13-444d-b6a7-ee3c7df42c06 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:50.732701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7182-ef13-444d-b6a7-ee3c7df42c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:50.732968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7182-ef13-444d-b6a7-ee3c7df42c06 None admin] PATH_INFO: `/v3/projects/92013b606d324e169a80498cfdc68734` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:50.734131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7182-ef13-444d-b6a7-ee3c7df42c06 None admin] RBAC: Authorizing `identity:get_project(project_id=92013b606d324e169a80498cfdc68734)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:50.735607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7182-ef13-444d-b6a7-ee3c7df42c06 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:50.737248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 564/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:06:50 2022] GET /identity/v3/projects/92013b606d324e169a80498cfdc68734 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:50.743792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f72a7a-c687-4ddc-9ee9-b21a78ef26e7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:50.848656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f72a7a-c687-4ddc-9ee9-b21a78ef26e7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:50.884207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e301dad-2ee3-4850-a36e-bd8703a540c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:50.884813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301dad-2ee3-4850-a36e-bd8703a540c6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:50.885570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301dad-2ee3-4850-a36e-bd8703a540c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:50.885570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301dad-2ee3-4850-a36e-bd8703a540c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:50.886353 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e301dad-2ee3-4850-a36e-bd8703a540c6 None admin] RBAC: Adding query filter params (role.id=0f4eb127111b4fdabec35268db5bde9e, scope.project.id=92013b606d324e169a80498cfdc68734, user.id=6ff1253ebd3d4ccb967c8c9b1c209d8b) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:50.886664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e301dad-2ee3-4850-a36e-bd8703a540c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:50.888123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e301dad-2ee3-4850-a36e-bd8703a540c6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:50.894041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 566/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:50 2022] GET /identity/v3/role_assignments?user.id=6ff1253ebd3d4ccb967c8c9b1c209d8b&scope.project.id=92013b606d324e169a80498cfdc68734&role.id=0f4eb127111b4fdabec35268db5bde9e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:52.841500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5429d35-ceac-4acd-8f67-bd5b28a5629a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:52.842061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5429d35-ceac-4acd-8f67-bd5b28a5629a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:52.842061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5429d35-ceac-4acd-8f67-bd5b28a5629a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:52.843414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 565/1131] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:52.849074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f275-1fa0-41fb-897a-512e4cb6d075 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:52.849508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f275-1fa0-41fb-897a-512e4cb6d075 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:52.849620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f275-1fa0-41fb-897a-512e4cb6d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:52.869039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3fb2f275-1fa0-41fb-897a-512e4cb6d075 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:52.870415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3fb2f275-1fa0-41fb-897a-512e4cb6d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:52.922000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:52.942829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 567/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:53.207031 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5429d35-ceac-4acd-8f67-bd5b28a5629a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:53.302113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:53.313469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5429d35-ceac-4acd-8f67-bd5b28a5629a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:53.351769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38e36d-86db-4901-a9e9-3bdb4f0ec80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:53.352399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e36d-86db-4901-a9e9-3bdb4f0ec80b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:53.352681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e36d-86db-4901-a9e9-3bdb4f0ec80b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:53.352951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e36d-86db-4901-a9e9-3bdb4f0ec80b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:53.354162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38e36d-86db-4901-a9e9-3bdb4f0ec80b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:53.355704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38e36d-86db-4901-a9e9-3bdb4f0ec80b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:53.357289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 566/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:53.363519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2f275-1fa0-41fb-897a-512e4cb6d075 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:53.467115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2f275-1fa0-41fb-897a-512e4cb6d075 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:53.505646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01930eb6-6c9b-4fd8-8d84-9ba0c0a67102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:53.506272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01930eb6-6c9b-4fd8-8d84-9ba0c0a67102 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:53.506545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01930eb6-6c9b-4fd8-8d84-9ba0c0a67102 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:53.506813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01930eb6-6c9b-4fd8-8d84-9ba0c0a67102 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:53.507492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01930eb6-6c9b-4fd8-8d84-9ba0c0a67102 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:53.508928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01930eb6-6c9b-4fd8-8d84-9ba0c0a67102 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:53.539537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 568/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:06:53 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:55.464143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81cdff-2f35-4453-922a-19bb0d5ddc82 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:55.464758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81cdff-2f35-4453-922a-19bb0d5ddc82 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:55.464758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81cdff-2f35-4453-922a-19bb0d5ddc82 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:55.466033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 567/1135] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:55.472069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643feb-9a02-4125-9133-6eaeae08eb32 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:55.472557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643feb-9a02-4125-9133-6eaeae08eb32 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:55.472557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643feb-9a02-4125-9133-6eaeae08eb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:55.491210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-62643feb-9a02-4125-9133-6eaeae08eb32 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:55.492423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-62643feb-9a02-4125-9133-6eaeae08eb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:55.520177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:55.563125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 569/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:55.833253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81cdff-2f35-4453-922a-19bb0d5ddc82 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:55.940198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81cdff-2f35-4453-922a-19bb0d5ddc82 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:55.975369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8fdce-36bb-4100-b3f0-866f5cf15da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:55.976078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8fdce-36bb-4100-b3f0-866f5cf15da1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:55.976403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8fdce-36bb-4100-b3f0-866f5cf15da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:55.976717 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8fdce-36bb-4100-b3f0-866f5cf15da1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:55.977881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8fdce-36bb-4100-b3f0-866f5cf15da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:55.979418 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8fdce-36bb-4100-b3f0-866f5cf15da1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:55.980977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 568/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:06:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:55.987050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62643feb-9a02-4125-9133-6eaeae08eb32 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:56.093699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62643feb-9a02-4125-9133-6eaeae08eb32 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:56.134698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855bb8a-912b-4363-b5d3-3ca1c3011cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:56.135387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855bb8a-912b-4363-b5d3-3ca1c3011cc6 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:56.135668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855bb8a-912b-4363-b5d3-3ca1c3011cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:56.135943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855bb8a-912b-4363-b5d3-3ca1c3011cc6 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:56.136610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855bb8a-912b-4363-b5d3-3ca1c3011cc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:56.138069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855bb8a-912b-4363-b5d3-3ca1c3011cc6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:56.161754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 570/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:06:55 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:06:58.100390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0880-6f70-4a96-9521-281f405339b6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:58.100979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0880-6f70-4a96-9521-281f405339b6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:58.100979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0880-6f70-4a96-9521-281f405339b6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:58.102286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 569/1139] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:06:58.107746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4cd8-0372-44e4-8d3a-2d649d50df22 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:58.108198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4cd8-0372-44e4-8d3a-2d649d50df22 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:58.108297 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4cd8-0372-44e4-8d3a-2d649d50df22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:58.126582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e3eb4cd8-0372-44e4-8d3a-2d649d50df22 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:06:58.127798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e3eb4cd8-0372-44e4-8d3a-2d649d50df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:06:58.142452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:58.197665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 571/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:06:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:06:58.458272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c0880-6f70-4a96-9521-281f405339b6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:58.491047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:58.578161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c0880-6f70-4a96-9521-281f405339b6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:58.614901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe5a8f-c93f-4108-a9a8-84e1a27a0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:58.615549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5a8f-c93f-4108-a9a8-84e1a27a0979 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:58.615825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5a8f-c93f-4108-a9a8-84e1a27a0979 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:58.616091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5a8f-c93f-4108-a9a8-84e1a27a0979 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:58.627102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe5a8f-c93f-4108-a9a8-84e1a27a0979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1dbe5a8f-c93f-4108-a9a8-84e1a27a0979 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:06:58.631812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:06:58.632956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:06:58.634259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 570/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:06:58.639205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb4cd8-0372-44e4-8d3a-2d649d50df22 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:58.738674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb4cd8-0372-44e4-8d3a-2d649d50df22 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:58.773901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:58.774528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:58.774803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:58.775068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:58.775850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:58.776153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:58.777511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:58.785556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 572/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:58.791727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe5a8f-c93f-4108-a9a8-84e1a27a0979 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:58.882146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:58.896127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe5a8f-c93f-4108-a9a8-84e1a27a0979 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:58.932241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233103b-fb24-43e9-9ed5-9190d8a33f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:58.932861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233103b-fb24-43e9-9ed5-9190d8a33f6a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:58.933135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233103b-fb24-43e9-9ed5-9190d8a33f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:58.933401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233103b-fb24-43e9-9ed5-9190d8a33f6a None admin] PATH_INFO: `/v3/users/94ebe943532f4fb694fbdb43946adc43` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:58.946734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233103b-fb24-43e9-9ed5-9190d8a33f6a None admin] RBAC: Authorizing `identity:get_user(user_id=94ebe943532f4fb694fbdb43946adc43)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:58.948265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233103b-fb24-43e9-9ed5-9190d8a33f6a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:58.949874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 571/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:06:58 2022] GET /identity/v3/users/94ebe943532f4fb694fbdb43946adc43 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:58.955995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:58.972857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:59.062847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68869b27-0f1b-412b-993c-8131931902d3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:59.100293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f639c75-00a2-4f93-96ef-394a9916b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:59.100892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639c75-00a2-4f93-96ef-394a9916b224 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:59.101168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639c75-00a2-4f93-96ef-394a9916b224 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:59.101431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639c75-00a2-4f93-96ef-394a9916b224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:59.168332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f639c75-00a2-4f93-96ef-394a9916b224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:59.169789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f639c75-00a2-4f93-96ef-394a9916b224 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:59.183778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:59.271906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 573/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:06:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:06:59.278550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233103b-fb24-43e9-9ed5-9190d8a33f6a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:59.379521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233103b-fb24-43e9-9ed5-9190d8a33f6a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:59.416348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48323459-ada6-41f2-a1c2-970bfddee679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:59.417054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48323459-ada6-41f2-a1c2-970bfddee679 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:59.417331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48323459-ada6-41f2-a1c2-970bfddee679 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:59.417598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48323459-ada6-41f2-a1c2-970bfddee679 None admin] PATH_INFO: `/v3/projects/ba6e36ccdb9b4aadaa997792dfefe89c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:59.418780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48323459-ada6-41f2-a1c2-970bfddee679 None admin] RBAC: Authorizing `identity:get_project(project_id=ba6e36ccdb9b4aadaa997792dfefe89c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:59.420243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48323459-ada6-41f2-a1c2-970bfddee679 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:59.421913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 572/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:06:59 2022] GET /identity/v3/projects/ba6e36ccdb9b4aadaa997792dfefe89c => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:06:59.428346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f639c75-00a2-4f93-96ef-394a9916b224 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:06:59.529048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f639c75-00a2-4f93-96ef-394a9916b224 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:06:59.542524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:06:59.568145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db35f1-1717-4dcb-b570-4798cc12c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:06:59.568745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db35f1-1717-4dcb-b570-4798cc12c159 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:06:59.569015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db35f1-1717-4dcb-b570-4798cc12c159 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:06:59.569273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db35f1-1717-4dcb-b570-4798cc12c159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:06:59.569988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db35f1-1717-4dcb-b570-4798cc12c159 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=ba6e36ccdb9b4aadaa997792dfefe89c, user.id=94ebe943532f4fb694fbdb43946adc43) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:06:59.570282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db35f1-1717-4dcb-b570-4798cc12c159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:06:59.571721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db35f1-1717-4dcb-b570-4798cc12c159 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:06:59.577223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 574/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:06:59 2022] GET /identity/v3/role_assignments?user.id=94ebe943532f4fb694fbdb43946adc43&scope.project.id=ba6e36ccdb9b4aadaa997792dfefe89c&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:01.481152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517593b-341a-4685-b2e7-7d660738a03b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:01.481770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517593b-341a-4685-b2e7-7d660738a03b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:01.481770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517593b-341a-4685-b2e7-7d660738a03b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:01.483032 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 573/1147] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:01.488427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618822c-6cd7-4348-90fc-ccd9b9c2aa09 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:01.488847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618822c-6cd7-4348-90fc-ccd9b9c2aa09 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:01.488937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618822c-6cd7-4348-90fc-ccd9b9c2aa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:01.508416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2618822c-6cd7-4348-90fc-ccd9b9c2aa09 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:01.509649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2618822c-6cd7-4348-90fc-ccd9b9c2aa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:01.581309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 575/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:07:01.848724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517593b-341a-4685-b2e7-7d660738a03b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:01.865469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:01.956023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517593b-341a-4685-b2e7-7d660738a03b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:01.991553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cba03f-a0fc-411c-8823-f77aa8b4da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:01.992175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cba03f-a0fc-411c-8823-f77aa8b4da68 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:01.992453 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cba03f-a0fc-411c-8823-f77aa8b4da68 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:01.992723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cba03f-a0fc-411c-8823-f77aa8b4da68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:02.003317 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cba03f-a0fc-411c-8823-f77aa8b4da68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d8cba03f-a0fc-411c-8823-f77aa8b4da68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:07:02.008073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:07:02.009125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:07:02.010158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 574/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:07:02.015822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618822c-6cd7-4348-90fc-ccd9b9c2aa09 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:02.119560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618822c-6cd7-4348-90fc-ccd9b9c2aa09 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:02.127151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:02.156046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:02.156662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:02.156938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:02.157208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:02.157868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:02.158260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:02.159583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:02.168915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 576/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:07:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:02.175042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cba03f-a0fc-411c-8823-f77aa8b4da68 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:02.253368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:02.280065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cba03f-a0fc-411c-8823-f77aa8b4da68 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:02.315845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f675c32-710a-4310-80b9-4c5ffc962e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:02.316478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f675c32-710a-4310-80b9-4c5ffc962e64 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:02.316754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f675c32-710a-4310-80b9-4c5ffc962e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:02.317029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f675c32-710a-4310-80b9-4c5ffc962e64 None admin] PATH_INFO: `/v3/users/94ebe943532f4fb694fbdb43946adc43` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:02.318193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f675c32-710a-4310-80b9-4c5ffc962e64 None admin] RBAC: Authorizing `identity:get_user(user_id=94ebe943532f4fb694fbdb43946adc43)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:02.319687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f675c32-710a-4310-80b9-4c5ffc962e64 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:02.321293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 575/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:07:02 2022] GET /identity/v3/users/94ebe943532f4fb694fbdb43946adc43 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:02.327607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:02.427956 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d1b1a-8e41-4770-a87c-dbad858a382d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:02.463440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0477a-f74f-4369-91c7-fcbdc2dcbe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:02.464071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0477a-f74f-4369-91c7-fcbdc2dcbe96 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:02.464345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0477a-f74f-4369-91c7-fcbdc2dcbe96 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:02.464614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0477a-f74f-4369-91c7-fcbdc2dcbe96 None admin] PATH_INFO: `/v3/projects/ba6e36ccdb9b4aadaa997792dfefe89c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:02.465870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0477a-f74f-4369-91c7-fcbdc2dcbe96 None admin] RBAC: Authorizing `identity:get_project(project_id=ba6e36ccdb9b4aadaa997792dfefe89c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:02.467348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0477a-f74f-4369-91c7-fcbdc2dcbe96 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:02.469006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 577/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:07:02 2022] GET /identity/v3/projects/ba6e36ccdb9b4aadaa997792dfefe89c => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:02.475519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f675c32-710a-4310-80b9-4c5ffc962e64 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:02.577423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f675c32-710a-4310-80b9-4c5ffc962e64 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:02.614400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684749d-d9ca-446f-8bcb-e2a7184b2133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:02.615054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684749d-d9ca-446f-8bcb-e2a7184b2133 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:02.615361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684749d-d9ca-446f-8bcb-e2a7184b2133 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:02.615628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684749d-d9ca-446f-8bcb-e2a7184b2133 None admin] PATH_INFO: `/v3/projects/ba6e36ccdb9b4aadaa997792dfefe89c/users/94ebe943532f4fb694fbdb43946adc43/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:02.617744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684749d-d9ca-446f-8bcb-e2a7184b2133 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6e36ccdb9b4aadaa997792dfefe89c, user_id=94ebe943532f4fb694fbdb43946adc43, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:02.619192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684749d-d9ca-446f-8bcb-e2a7184b2133 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:02.634198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 576/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:07:02 2022] PUT /identity/v3/projects/ba6e36ccdb9b4aadaa997792dfefe89c/users/94ebe943532f4fb694fbdb43946adc43/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:07:04.529938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da871a-4103-44ab-bc97-754f8562f769 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:04.530500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da871a-4103-44ab-bc97-754f8562f769 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:04.530500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da871a-4103-44ab-bc97-754f8562f769 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:04.531838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 578/1154] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:04.537306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275ccc-dee6-41a8-a102-8263e9efe0d7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:04.537750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275ccc-dee6-41a8-a102-8263e9efe0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:04.537858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275ccc-dee6-41a8-a102-8263e9efe0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:04.544354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:04.559672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fe275ccc-dee6-41a8-a102-8263e9efe0d7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:04.560875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fe275ccc-dee6-41a8-a102-8263e9efe0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:04.633865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 577/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:07:04.903728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da871a-4103-44ab-bc97-754f8562f769 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:04.940518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:05.016247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da871a-4103-44ab-bc97-754f8562f769 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:05.054414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055c16-44d4-41b2-b8c3-79558062fa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:05.055039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055c16-44d4-41b2-b8c3-79558062fa2c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:05.055341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055c16-44d4-41b2-b8c3-79558062fa2c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:05.055603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055c16-44d4-41b2-b8c3-79558062fa2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:05.066645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60055c16-44d4-41b2-b8c3-79558062fa2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-60055c16-44d4-41b2-b8c3-79558062fa2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:07:05.071381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:07:05.072486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:07:05.073487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 579/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:07:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:07:05.079000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe275ccc-dee6-41a8-a102-8263e9efe0d7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:05.184492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe275ccc-dee6-41a8-a102-8263e9efe0d7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:05.214989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:05.223030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:05.223788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:05.224071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:05.224344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:05.225019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:05.225320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:05.226990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:05.236262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 578/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:05.242327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055c16-44d4-41b2-b8c3-79558062fa2c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:05.339903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:05.347263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055c16-44d4-41b2-b8c3-79558062fa2c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:05.382002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648da0c-3b7e-4d12-8f9b-d68c5ad00898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:05.382623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648da0c-3b7e-4d12-8f9b-d68c5ad00898 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:05.382894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648da0c-3b7e-4d12-8f9b-d68c5ad00898 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:05.383162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648da0c-3b7e-4d12-8f9b-d68c5ad00898 None admin] PATH_INFO: `/v3/users/94ebe943532f4fb694fbdb43946adc43` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:05.384339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648da0c-3b7e-4d12-8f9b-d68c5ad00898 None admin] RBAC: Authorizing `identity:get_user(user_id=94ebe943532f4fb694fbdb43946adc43)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:05.385809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648da0c-3b7e-4d12-8f9b-d68c5ad00898 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:05.387434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 580/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:07:05 2022] GET /identity/v3/users/94ebe943532f4fb694fbdb43946adc43 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:05.393682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:05.513864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80daa738-8f25-4f33-8a2b-d0bf8dad2984 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:05.553693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cfce9-fe73-48ef-a5a3-acc147b35084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:05.554431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cfce9-fe73-48ef-a5a3-acc147b35084 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:05.554526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cfce9-fe73-48ef-a5a3-acc147b35084 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:05.554807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cfce9-fe73-48ef-a5a3-acc147b35084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:05.618174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:05.627126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cfce9-fe73-48ef-a5a3-acc147b35084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:05.628630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cfce9-fe73-48ef-a5a3-acc147b35084 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:05.733133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 579/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:07:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:07:05.739516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648da0c-3b7e-4d12-8f9b-d68c5ad00898 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:05.847789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648da0c-3b7e-4d12-8f9b-d68c5ad00898 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:05.885227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd60a7e-fb28-4a8c-819f-6d7d629a3e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:05.885928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60a7e-fb28-4a8c-819f-6d7d629a3e13 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:05.886192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60a7e-fb28-4a8c-819f-6d7d629a3e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:05.886449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60a7e-fb28-4a8c-819f-6d7d629a3e13 None admin] PATH_INFO: `/v3/projects/ba6e36ccdb9b4aadaa997792dfefe89c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:05.887677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd60a7e-fb28-4a8c-819f-6d7d629a3e13 None admin] RBAC: Authorizing `identity:get_project(project_id=ba6e36ccdb9b4aadaa997792dfefe89c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:05.889115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd60a7e-fb28-4a8c-819f-6d7d629a3e13 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:05.890785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 581/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:07:05 2022] GET /identity/v3/projects/ba6e36ccdb9b4aadaa997792dfefe89c => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:05.897365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cfce9-fe73-48ef-a5a3-acc147b35084 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:06.006881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:06.014739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cfce9-fe73-48ef-a5a3-acc147b35084 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:06.053118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852ba19-01e5-45c9-9e76-b484660e6470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:06.053716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852ba19-01e5-45c9-9e76-b484660e6470 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:06.053987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852ba19-01e5-45c9-9e76-b484660e6470 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:06.054253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852ba19-01e5-45c9-9e76-b484660e6470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:06.054974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852ba19-01e5-45c9-9e76-b484660e6470 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=ba6e36ccdb9b4aadaa997792dfefe89c, user.id=94ebe943532f4fb694fbdb43946adc43) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:06.055280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852ba19-01e5-45c9-9e76-b484660e6470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:06.057135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852ba19-01e5-45c9-9e76-b484660e6470 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:06.063210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 580/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:07:05 2022] GET /identity/v3/role_assignments?user.id=94ebe943532f4fb694fbdb43946adc43&scope.project.id=ba6e36ccdb9b4aadaa997792dfefe89c&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:07:07.981849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849e7d7-3a4a-49ea-a72c-fec096983d13 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:07.982399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849e7d7-3a4a-49ea-a72c-fec096983d13 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:07.982399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849e7d7-3a4a-49ea-a72c-fec096983d13 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:07.983770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 582/1162] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:07.989172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c10f4e-da54-49f6-9fc1-ab32db1421d1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:07.989609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c10f4e-da54-49f6-9fc1-ab32db1421d1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:07.989697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c10f4e-da54-49f6-9fc1-ab32db1421d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:08.010735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-74c10f4e-da54-49f6-9fc1-ab32db1421d1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:08.011991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-74c10f4e-da54-49f6-9fc1-ab32db1421d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:08.085098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 581/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:07:08.344871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849e7d7-3a4a-49ea-a72c-fec096983d13 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:08.378653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:08.452384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849e7d7-3a4a-49ea-a72c-fec096983d13 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:08.488209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800df0a7-a2bd-40f1-9bbe-306800cd8b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:08.488817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df0a7-a2bd-40f1-9bbe-306800cd8b68 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:08.489095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df0a7-a2bd-40f1-9bbe-306800cd8b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:08.489363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df0a7-a2bd-40f1-9bbe-306800cd8b68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:08.560496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800df0a7-a2bd-40f1-9bbe-306800cd8b68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:08.561945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800df0a7-a2bd-40f1-9bbe-306800cd8b68 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:08.592998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:08.672330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 583/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:07:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:07:08.678565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c10f4e-da54-49f6-9fc1-ab32db1421d1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:08.770537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:08.784140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c10f4e-da54-49f6-9fc1-ab32db1421d1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:08.824596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a730aa-0807-410e-8861-754189d1343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:08.825173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a730aa-0807-410e-8861-754189d1343c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:08.825411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a730aa-0807-410e-8861-754189d1343c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:08.825596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a730aa-0807-410e-8861-754189d1343c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:08.832736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a730aa-0807-410e-8861-754189d1343c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:08.834191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a730aa-0807-410e-8861-754189d1343c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-58a730aa-0807-410e-8861-754189d1343c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:07:08.845782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:07:08.847408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:07:08.848273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 582/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:07:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:07:08.854186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800df0a7-a2bd-40f1-9bbe-306800cd8b68 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:08.965859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800df0a7-a2bd-40f1-9bbe-306800cd8b68 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:08.997797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:09.004037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c3d2d-c450-4a84-a6f1-c4f8e15fc5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:09.004710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c3d2d-c450-4a84-a6f1-c4f8e15fc5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:09.005034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c3d2d-c450-4a84-a6f1-c4f8e15fc5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:09.005367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c3d2d-c450-4a84-a6f1-c4f8e15fc5f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:09.006157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c3d2d-c450-4a84-a6f1-c4f8e15fc5f7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:09.006527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c3d2d-c450-4a84-a6f1-c4f8e15fc5f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:09.008012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c3d2d-c450-4a84-a6f1-c4f8e15fc5f7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:09.016641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 584/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 31 12:07:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:10.929203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a3e63-315e-4416-ab09-ac14e501435d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:10.929822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a3e63-315e-4416-ab09-ac14e501435d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:10.929822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a3e63-315e-4416-ab09-ac14e501435d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:10.931405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 583/1167] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:10.938073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c56ec-5eae-43d8-8da2-bc5b9b73a620 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:10.938634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c56ec-5eae-43d8-8da2-bc5b9b73a620 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:10.938982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c56ec-5eae-43d8-8da2-bc5b9b73a620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:10.968096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8e9c56ec-5eae-43d8-8da2-bc5b9b73a620 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:10.970065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8e9c56ec-5eae-43d8-8da2-bc5b9b73a620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:11.090536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 585/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:07:11.356841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a3e63-315e-4416-ab09-ac14e501435d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:11.484489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a3e63-315e-4416-ab09-ac14e501435d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:11.520834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e829af-d553-4210-88d6-981faa1e0ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:11.521444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e829af-d553-4210-88d6-981faa1e0ae9 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:11.521721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e829af-d553-4210-88d6-981faa1e0ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:11.521988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e829af-d553-4210-88d6-981faa1e0ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:11.522672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e829af-d553-4210-88d6-981faa1e0ae9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:11.524088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e829af-d553-4210-88d6-981faa1e0ae9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:11.547904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 584/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Wed Aug 31 12:07:11 2022] POST /identity/v3/domains => generated 267 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:07:13.556447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97e354-15ff-4418-aeb8-c817c0d3aa67 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:13.557288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97e354-15ff-4418-aeb8-c817c0d3aa67 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:13.557288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97e354-15ff-4418-aeb8-c817c0d3aa67 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:13.558484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 586/1170] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:13.564226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3974696-89c0-494b-a0a0-e24b366da2f4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:13.564794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3974696-89c0-494b-a0a0-e24b366da2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:13.564794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3974696-89c0-494b-a0a0-e24b366da2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:13.585272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b3974696-89c0-494b-a0a0-e24b366da2f4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:13.586462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b3974696-89c0-494b-a0a0-e24b366da2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:13.588737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:13.660059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 585/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:07:13.944736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97e354-15ff-4418-aeb8-c817c0d3aa67 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:14.070064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97e354-15ff-4418-aeb8-c817c0d3aa67 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:14.102595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:14.113882 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7c73a-ca32-4acc-addd-4d1674db5919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:14.114542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7c73a-ca32-4acc-addd-4d1674db5919 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:14.114816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7c73a-ca32-4acc-addd-4d1674db5919 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:14.115085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7c73a-ca32-4acc-addd-4d1674db5919 None admin] PATH_INFO: `/v3/domains/5c1a784262194723b1ff66271f8f12e7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:14.128028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7c73a-ca32-4acc-addd-4d1674db5919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5c1a784262194723b1ff66271f8f12e7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:14.129543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7c73a-ca32-4acc-addd-4d1674db5919 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:14.131321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 587/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Aug 31 12:07:13 2022] GET /identity/v3/domains/5c1a784262194723b1ff66271f8f12e7 => generated 267 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:14.138085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3974696-89c0-494b-a0a0-e24b366da2f4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:14.247838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3974696-89c0-494b-a0a0-e24b366da2f4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:14.287221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e64e52-9249-4bbd-a4d3-80ebb9744c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:14.287934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e64e52-9249-4bbd-a4d3-80ebb9744c82 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:14.288250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e64e52-9249-4bbd-a4d3-80ebb9744c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:14.288565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e64e52-9249-4bbd-a4d3-80ebb9744c82 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:14.289264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e64e52-9249-4bbd-a4d3-80ebb9744c82 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:14.291017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e64e52-9249-4bbd-a4d3-80ebb9744c82 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:14.301428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:14.319910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 586/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:07:14 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:07:16.348317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f4bb-a219-43c5-b3e5-9f4e94944a36 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:16.348905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f4bb-a219-43c5-b3e5-9f4e94944a36 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:16.348905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f4bb-a219-43c5-b3e5-9f4e94944a36 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:16.350321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 588/1174] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:16.356250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62483e67-711c-44c2-8a85-33982123791d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:16.356774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62483e67-711c-44c2-8a85-33982123791d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:16.356888 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62483e67-711c-44c2-8a85-33982123791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:16.376195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-62483e67-711c-44c2-8a85-33982123791d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:16.377392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-62483e67-711c-44c2-8a85-33982123791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:16.448646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 587/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:07:16.713347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79f4bb-a219-43c5-b3e5-9f4e94944a36 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:16.820293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79f4bb-a219-43c5-b3e5-9f4e94944a36 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:16.856375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411c5a4-397d-44a9-965b-9f88fad6ded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:16.857007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c5a4-397d-44a9-965b-9f88fad6ded8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:16.857281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c5a4-397d-44a9-965b-9f88fad6ded8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:16.857548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c5a4-397d-44a9-965b-9f88fad6ded8 None admin] PATH_INFO: `/v3/domains/5c1a784262194723b1ff66271f8f12e7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:16.858739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411c5a4-397d-44a9-965b-9f88fad6ded8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5c1a784262194723b1ff66271f8f12e7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:16.860219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411c5a4-397d-44a9-965b-9f88fad6ded8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:16.861817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 589/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Aug 31 12:07:16 2022] GET /identity/v3/domains/5c1a784262194723b1ff66271f8f12e7 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:16.867996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62483e67-711c-44c2-8a85-33982123791d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:16.977705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62483e67-711c-44c2-8a85-33982123791d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:17.001088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:17.017259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34afb4f-c935-4900-9436-253829e78d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:17.017883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34afb4f-c935-4900-9436-253829e78d27 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:17.018155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34afb4f-c935-4900-9436-253829e78d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:17.018416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34afb4f-c935-4900-9436-253829e78d27 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:17.019064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34afb4f-c935-4900-9436-253829e78d27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:17.020548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34afb4f-c935-4900-9436-253829e78d27 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:17.048740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 588/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:07:16 2022] POST /identity/v3/users => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:07:19.003207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385024fb-aa03-4e9a-b9b3-dfabeb1564e8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:19.003826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385024fb-aa03-4e9a-b9b3-dfabeb1564e8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:19.003826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385024fb-aa03-4e9a-b9b3-dfabeb1564e8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:19.005077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 590/1178] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:19.011002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b155-8fb8-4d72-baaa-d32aedf899f4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:19.011504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b155-8fb8-4d72-baaa-d32aedf899f4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:19.011597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b155-8fb8-4d72-baaa-d32aedf899f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:19.030867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9a36b155-8fb8-4d72-baaa-d32aedf899f4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:19.032097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9a36b155-8fb8-4d72-baaa-d32aedf899f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:19.103191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 589/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:07:19.367119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385024fb-aa03-4e9a-b9b3-dfabeb1564e8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:19.428527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:19.485270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385024fb-aa03-4e9a-b9b3-dfabeb1564e8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:19.520957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e13c-a47c-4552-b7d5-6642be12197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:19.521578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e13c-a47c-4552-b7d5-6642be12197e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:19.521848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e13c-a47c-4552-b7d5-6642be12197e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:19.522110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e13c-a47c-4552-b7d5-6642be12197e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:19.532631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982e13c-a47c-4552-b7d5-6642be12197e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6982e13c-a47c-4552-b7d5-6642be12197e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:07:19.537358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:07:19.538516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:07:19.539653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 591/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:07:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:07:19.545263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36b155-8fb8-4d72-baaa-d32aedf899f4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:19.649104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36b155-8fb8-4d72-baaa-d32aedf899f4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:19.656301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:19.683872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:19.684466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:19.684736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:19.684991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:19.685638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:19.685929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:19.687391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:19.695317 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 590/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:07:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:19.701589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e13c-a47c-4552-b7d5-6642be12197e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:19.807850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e13c-a47c-4552-b7d5-6642be12197e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:19.823597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:19.845805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7f519-204a-42c5-bceb-9af00606657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:19.846750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7f519-204a-42c5-bceb-9af00606657d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:19.847029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7f519-204a-42c5-bceb-9af00606657d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:19.847331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7f519-204a-42c5-bceb-9af00606657d None admin] PATH_INFO: `/v3/users/c083ec699ee1484ab71ee94d274ca658` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:19.862172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7f519-204a-42c5-bceb-9af00606657d None admin] RBAC: Authorizing `identity:get_user(user_id=c083ec699ee1484ab71ee94d274ca658)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:19.863663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7f519-204a-42c5-bceb-9af00606657d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:19.865345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 592/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:07:19 2022] GET /identity/v3/users/c083ec699ee1484ab71ee94d274ca658 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:19.871672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:19.981200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c9c24-aef4-4199-b6bd-ee7a5136f153 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:20.020364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c7e18-d1be-41ec-8b23-ddac89645979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:20.021039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c7e18-d1be-41ec-8b23-ddac89645979 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:20.021232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c7e18-d1be-41ec-8b23-ddac89645979 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:20.021501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c7e18-d1be-41ec-8b23-ddac89645979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:20.057304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:20.092159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c7e18-d1be-41ec-8b23-ddac89645979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:20.093618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c7e18-d1be-41ec-8b23-ddac89645979 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:20.196412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 591/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:07:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:07:20.202990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7f519-204a-42c5-bceb-9af00606657d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:20.319670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7f519-204a-42c5-bceb-9af00606657d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:20.356645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cda5a-e847-47fd-bc44-0dbda0e1b0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:20.357292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cda5a-e847-47fd-bc44-0dbda0e1b0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:20.357560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cda5a-e847-47fd-bc44-0dbda0e1b0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:20.357830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cda5a-e847-47fd-bc44-0dbda0e1b0f0 None admin] PATH_INFO: `/v3/projects/ea83e24548cd4e73a7e600f715ce00f6` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:20.359146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cda5a-e847-47fd-bc44-0dbda0e1b0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=ea83e24548cd4e73a7e600f715ce00f6)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:20.360617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cda5a-e847-47fd-bc44-0dbda0e1b0f0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:20.362404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 593/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:07:20 2022] GET /identity/v3/projects/ea83e24548cd4e73a7e600f715ce00f6 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:20.369348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c7e18-d1be-41ec-8b23-ddac89645979 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:20.438718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:20.482861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c7e18-d1be-41ec-8b23-ddac89645979 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:20.522092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700085d-78e0-473e-befb-ba583474ff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:20.522694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700085d-78e0-473e-befb-ba583474ff65 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:20.522967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700085d-78e0-473e-befb-ba583474ff65 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:20.523272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700085d-78e0-473e-befb-ba583474ff65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:20.523977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700085d-78e0-473e-befb-ba583474ff65 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=ea83e24548cd4e73a7e600f715ce00f6, user.id=c083ec699ee1484ab71ee94d274ca658) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:20.524277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700085d-78e0-473e-befb-ba583474ff65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:20.526213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700085d-78e0-473e-befb-ba583474ff65 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:20.532183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 592/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:07:20 2022] GET /identity/v3/role_assignments?user.id=c083ec699ee1484ab71ee94d274ca658&scope.project.id=ea83e24548cd4e73a7e600f715ce00f6&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:22.432454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca8f86-f2ba-4bb1-a259-28c8728bca0c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:22.433052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca8f86-f2ba-4bb1-a259-28c8728bca0c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:22.433052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca8f86-f2ba-4bb1-a259-28c8728bca0c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:22.434498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 594/1186] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:22.440722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5b2de-2309-4a8d-816d-3305a8a4970e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:22.441285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5b2de-2309-4a8d-816d-3305a8a4970e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:22.441285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5b2de-2309-4a8d-816d-3305a8a4970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:22.464075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-95a5b2de-2309-4a8d-816d-3305a8a4970e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:22.465758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95a5b2de-2309-4a8d-816d-3305a8a4970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:22.560869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 593/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:07:23.057618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca8f86-f2ba-4bb1-a259-28c8728bca0c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:23.105871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:23.170455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca8f86-f2ba-4bb1-a259-28c8728bca0c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:23.206800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3574bfe-7681-4874-b710-0a716c06f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:23.207454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3574bfe-7681-4874-b710-0a716c06f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:23.207729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3574bfe-7681-4874-b710-0a716c06f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:23.207996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3574bfe-7681-4874-b710-0a716c06f4c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:23.219172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3574bfe-7681-4874-b710-0a716c06f4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b3574bfe-7681-4874-b710-0a716c06f4c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:07:23.224580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:07:23.226352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:07:23.228327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 595/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:07:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:07:23.233881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5b2de-2309-4a8d-816d-3305a8a4970e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:23.300215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:23.343125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5b2de-2309-4a8d-816d-3305a8a4970e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:23.379699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:23.380370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:23.380685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:23.380993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:23.381733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:23.382067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:23.383568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:23.392411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 594/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:07:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:07:23.398501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3574bfe-7681-4874-b710-0a716c06f4c5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:23.500988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:23.504869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3574bfe-7681-4874-b710-0a716c06f4c5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:23.545239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87b806-d715-4ea1-8781-245f079d9014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:23.545875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87b806-d715-4ea1-8781-245f079d9014 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:23.546180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87b806-d715-4ea1-8781-245f079d9014 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:23.546444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87b806-d715-4ea1-8781-245f079d9014 None admin] PATH_INFO: `/v3/users/c083ec699ee1484ab71ee94d274ca658` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:23.547665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87b806-d715-4ea1-8781-245f079d9014 None admin] RBAC: Authorizing `identity:get_user(user_id=c083ec699ee1484ab71ee94d274ca658)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:23.549104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87b806-d715-4ea1-8781-245f079d9014 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:23.550709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 596/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:07:23 2022] GET /identity/v3/users/c083ec699ee1484ab71ee94d274ca658 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:23.559671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:23.680090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5f79f-cd05-4401-923c-5c5bf9895ad7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:23.706051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:23.718669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1191a6-d3e3-4b88-8714-307ab4c20705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:23.719405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1191a6-d3e3-4b88-8714-307ab4c20705 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:23.719731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1191a6-d3e3-4b88-8714-307ab4c20705 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:23.720062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1191a6-d3e3-4b88-8714-307ab4c20705 None admin] PATH_INFO: `/v3/projects/ea83e24548cd4e73a7e600f715ce00f6` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:23.721667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1191a6-d3e3-4b88-8714-307ab4c20705 None admin] RBAC: Authorizing `identity:get_project(project_id=ea83e24548cd4e73a7e600f715ce00f6)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:23.723201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1191a6-d3e3-4b88-8714-307ab4c20705 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:23.724944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 595/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:07:23 2022] GET /identity/v3/projects/ea83e24548cd4e73a7e600f715ce00f6 => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:23.731208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87b806-d715-4ea1-8781-245f079d9014 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:23.841072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87b806-d715-4ea1-8781-245f079d9014 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:23.879624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e231e96-445b-4e5a-998e-005d4277ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:23.880298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e231e96-445b-4e5a-998e-005d4277ee21 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:23.880572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e231e96-445b-4e5a-998e-005d4277ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:23.880828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e231e96-445b-4e5a-998e-005d4277ee21 None admin] PATH_INFO: `/v3/projects/ea83e24548cd4e73a7e600f715ce00f6/users/c083ec699ee1484ab71ee94d274ca658/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:23.882799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e231e96-445b-4e5a-998e-005d4277ee21 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea83e24548cd4e73a7e600f715ce00f6, user_id=c083ec699ee1484ab71ee94d274ca658, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:23.884406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e231e96-445b-4e5a-998e-005d4277ee21 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:23.900259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 597/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:07:23 2022] PUT /identity/v3/projects/ea83e24548cd4e73a7e600f715ce00f6/users/c083ec699ee1484ab71ee94d274ca658/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:07:25.894565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd916183-16c5-46a3-9835-cd9a816dd3e7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:25.895202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd916183-16c5-46a3-9835-cd9a816dd3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:25.895645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd916183-16c5-46a3-9835-cd9a816dd3e7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:25.897239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 596/1193] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:07:25.904377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b2774-586d-4637-a984-ba5f080fba8b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:25.905007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b2774-586d-4637-a984-ba5f080fba8b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:25.905007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b2774-586d-4637-a984-ba5f080fba8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:25.925096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-073b2774-586d-4637-a984-ba5f080fba8b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:07:25.926355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-073b2774-586d-4637-a984-ba5f080fba8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:07:25.955184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:26.008213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 598/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:07:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:07:26.276154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd916183-16c5-46a3-9835-cd9a816dd3e7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:26.393451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd916183-16c5-46a3-9835-cd9a816dd3e7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:26.435324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00308e4-3a9f-4cdc-85a5-46f07a46b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:26.435966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00308e4-3a9f-4cdc-85a5-46f07a46b811 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:26.436241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00308e4-3a9f-4cdc-85a5-46f07a46b811 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:26.436506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00308e4-3a9f-4cdc-85a5-46f07a46b811 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:26.450301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00308e4-3a9f-4cdc-85a5-46f07a46b811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c00308e4-3a9f-4cdc-85a5-46f07a46b811 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:07:26.455341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:07:26.457076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:07:26.458687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 597/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:07:26.466785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b2774-586d-4637-a984-ba5f080fba8b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:26.581003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b2774-586d-4637-a984-ba5f080fba8b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:26.618585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:26.619412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:26.619737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:26.620052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:26.620770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:26.621095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:26.622537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:26.632017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 599/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:07:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:26.638738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00308e4-3a9f-4cdc-85a5-46f07a46b811 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:26.696239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:26.759253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00308e4-3a9f-4cdc-85a5-46f07a46b811 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:26.797540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef74a7f-3788-4e4a-9aec-7801df8c8da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:26.798179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74a7f-3788-4e4a-9aec-7801df8c8da7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:26.798451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74a7f-3788-4e4a-9aec-7801df8c8da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:26.798716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74a7f-3788-4e4a-9aec-7801df8c8da7 None admin] PATH_INFO: `/v3/users/c083ec699ee1484ab71ee94d274ca658` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:26.799881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef74a7f-3788-4e4a-9aec-7801df8c8da7 None admin] RBAC: Authorizing `identity:get_user(user_id=c083ec699ee1484ab71ee94d274ca658)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:26.801333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef74a7f-3788-4e4a-9aec-7801df8c8da7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:26.802967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 598/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 31 12:07:26 2022] GET /identity/v3/users/c083ec699ee1484ab71ee94d274ca658 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:26.812274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:26.837797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:26.927515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ec47f-3f51-40c7-8091-7587f3df6dcf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:26.964775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422d438-70b5-45f0-9106-c7981bfe2c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:26.965444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422d438-70b5-45f0-9106-c7981bfe2c53 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:26.965760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422d438-70b5-45f0-9106-c7981bfe2c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:26.966094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422d438-70b5-45f0-9106-c7981bfe2c53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:27.043684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422d438-70b5-45f0-9106-c7981bfe2c53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:27.045279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422d438-70b5-45f0-9106-c7981bfe2c53 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:27.068145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:27.160842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 600/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:07:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:07:27.167613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef74a7f-3788-4e4a-9aec-7801df8c8da7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:27.282837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef74a7f-3788-4e4a-9aec-7801df8c8da7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:27.305784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:27.322257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d742427-4217-4a45-9369-e4d4ed8b2bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:27.322956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d742427-4217-4a45-9369-e4d4ed8b2bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:27.323290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d742427-4217-4a45-9369-e4d4ed8b2bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:27.323621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d742427-4217-4a45-9369-e4d4ed8b2bd2 None admin] PATH_INFO: `/v3/projects/ea83e24548cd4e73a7e600f715ce00f6` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:27.324924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d742427-4217-4a45-9369-e4d4ed8b2bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=ea83e24548cd4e73a7e600f715ce00f6)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:27.326445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d742427-4217-4a45-9369-e4d4ed8b2bd2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:27.328271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 599/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 31 12:07:27 2022] GET /identity/v3/projects/ea83e24548cd4e73a7e600f715ce00f6 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:27.335016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422d438-70b5-45f0-9106-c7981bfe2c53 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:07:27.449687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422d438-70b5-45f0-9106-c7981bfe2c53 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:07:27.473641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:27.491004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac62b53-fb58-428d-86a9-afdb425e4d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:07:27.491631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac62b53-fb58-428d-86a9-afdb425e4d0a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:07:27.491919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac62b53-fb58-428d-86a9-afdb425e4d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:07:27.492186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac62b53-fb58-428d-86a9-afdb425e4d0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:07:27.492949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac62b53-fb58-428d-86a9-afdb425e4d0a None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=ea83e24548cd4e73a7e600f715ce00f6, user.id=c083ec699ee1484ab71ee94d274ca658) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:07:27.493278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac62b53-fb58-428d-86a9-afdb425e4d0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:07:27.495097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac62b53-fb58-428d-86a9-afdb425e4d0a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:07:27.501719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 601/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:07:27 2022] GET /identity/v3/role_assignments?user.id=c083ec699ee1484ab71ee94d274ca658&scope.project.id=ea83e24548cd4e73a7e600f715ce00f6&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:07:37.318477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:07:37.499473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=8 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:27.254941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b3711-c320-43e9-b24b-6ffa3800c16b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:27.255955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b3711-c320-43e9-b24b-6ffa3800c16b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:27.255955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b3711-c320-43e9-b24b-6ffa3800c16b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:27.257320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 600/1201] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:08:27.264073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4e9e-741e-4cc8-8135-c7486f602517 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:27.264986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4e9e-741e-4cc8-8135-c7486f602517 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:27.264986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4e9e-741e-4cc8-8135-c7486f602517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:27.294769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8ecd4e9e-741e-4cc8-8135-c7486f602517 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:08:27.296170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8ecd4e9e-741e-4cc8-8135-c7486f602517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:08:27.408134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 602/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:08:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:08:27.687321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b3711-c320-43e9-b24b-6ffa3800c16b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:27.881626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b3711-c320-43e9-b24b-6ffa3800c16b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:27.945726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:27.947033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:27.947616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:27.948113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:27.966924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:27.969857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:27.994423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:27.995893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:28.000636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:28.000636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:28.000636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:28.000636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 601/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:28.005135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd4e9e-741e-4cc8-8135-c7486f602517 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:28.202250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd4e9e-741e-4cc8-8135-c7486f602517 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:28.246639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:28.253741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31f5dd-f675-4fba-9027-8eb95f8b1b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:28.254556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31f5dd-f675-4fba-9027-8eb95f8b1b9c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:28.254724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31f5dd-f675-4fba-9027-8eb95f8b1b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:28.254986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31f5dd-f675-4fba-9027-8eb95f8b1b9c None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:28.255939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31f5dd-f675-4fba-9027-8eb95f8b1b9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:28.256188 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31f5dd-f675-4fba-9027-8eb95f8b1b9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:28.258173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31f5dd-f675-4fba-9027-8eb95f8b1b9c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:28.291017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 603/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:28.297608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:28.398061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:28.489823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b34f86-9f67-4540-8edc-c71b0e0c2b54 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:28.546089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dde481-ea90-4b7a-8a33-7efbf6be2cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:28.547211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dde481-ea90-4b7a-8a33-7efbf6be2cd8 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:28.547712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dde481-ea90-4b7a-8a33-7efbf6be2cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:28.548142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dde481-ea90-4b7a-8a33-7efbf6be2cd8 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:28.549358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dde481-ea90-4b7a-8a33-7efbf6be2cd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:28.551967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dde481-ea90-4b7a-8a33-7efbf6be2cd8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:28.591374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 602/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 31 12:08:28 2022] POST /identity/v3/users => generated 251 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:08:31.213789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fc932-156a-4d84-8f54-dd8acdfb7e81 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:31.214410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fc932-156a-4d84-8f54-dd8acdfb7e81 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:31.214410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fc932-156a-4d84-8f54-dd8acdfb7e81 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:31.216512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 604/1206] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:08:31.222322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022af5f-0d30-41a6-9e7d-d568e37742df None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:31.222895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022af5f-0d30-41a6-9e7d-d568e37742df None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:31.222895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022af5f-0d30-41a6-9e7d-d568e37742df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:31.258935 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4022af5f-0d30-41a6-9e7d-d568e37742df None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:08:31.261316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4022af5f-0d30-41a6-9e7d-d568e37742df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:08:31.405762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 603/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:08:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:08:31.955714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fc932-156a-4d84-8f54-dd8acdfb7e81 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:32.148699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fc932-156a-4d84-8f54-dd8acdfb7e81 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:32.203990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758db33-89df-4aec-925b-d947f108cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:32.205072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758db33-89df-4aec-925b-d947f108cb89 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:32.205395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758db33-89df-4aec-925b-d947f108cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:32.206039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758db33-89df-4aec-925b-d947f108cb89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:32.227339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758db33-89df-4aec-925b-d947f108cb89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-4758db33-89df-4aec-925b-d947f108cb89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:08:32.236357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:32.237502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:32.239905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 605/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:08:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:32.244818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022af5f-0d30-41a6-9e7d-d568e37742df None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:32.313163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:32.404058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022af5f-0d30-41a6-9e7d-d568e37742df None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:32.459857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:32.460938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:32.461439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:32.461898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:32.462773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:32.463403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:32.465230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:32.485433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 604/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:08:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:32.486820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758db33-89df-4aec-925b-d947f108cb89 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:32.561868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:32.630930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758db33-89df-4aec-925b-d947f108cb89 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:32.681761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:32.682328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:32.682602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:32.682894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:32.696076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:32.697637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:32.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:32.720813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:32.723259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:32.723259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:32.723259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:32.725556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 606/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:32.726451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:32.848407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:32.852293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437c132-e170-48a5-bd67-00f3f1a1650e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:32.901308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:32.901926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:32.902283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:32.902532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:32.903294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:32.903578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:32.905202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:32.922631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 605/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:32.928468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:33.091500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215bf12-9034-4921-89b4-a09476e8ec45 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:33.093339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:33.140455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:33.141933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:33.142517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:33.142897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:33.153826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:33.155415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:08:33.184410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:33.185811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:33.186791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:33.187413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 607/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:33.196342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:33.329168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699f79-a7f2-446a-bccd-dd383bdcb80e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:33.374694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:33.375438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:33.375736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:33.376010 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:33.377063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:33.377498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:33.379263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:33.408117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 606/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 31 12:08:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:08:33.415921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:33.573865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4b16-3181-42e4-827f-fa30ff5e6e95 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:33.628899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b609d1-1c53-4841-83f3-c1a62402956b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:33.630614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b609d1-1c53-4841-83f3-c1a62402956b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:33.631204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b609d1-1c53-4841-83f3-c1a62402956b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:33.632619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b609d1-1c53-4841-83f3-c1a62402956b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:33.695789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:33.758889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b609d1-1c53-4841-83f3-c1a62402956b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:33.762937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b609d1-1c53-4841-83f3-c1a62402956b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:33.906603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 608/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:08:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:08:33.913548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:33.943472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:34.056564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12b16-c89a-43b4-99e1-082a615e4d74 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:34.122775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:34.123912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:34.124409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:34.124906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:34.145683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:34.148565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:34.177201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:34.178460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:34.179924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:34.179924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:34.179924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:34.179924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 607/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:34.189703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b609d1-1c53-4841-83f3-c1a62402956b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:34.299557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:34.372049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b609d1-1c53-4841-83f3-c1a62402956b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:34.441544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:34.442494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:34.442974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:34.443496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:34.444684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:34.445229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:34.447808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:34.466938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 609/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:34.477239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:34.599115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:34.637526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c572bd3-ff65-4014-a2b4-df695e07a193 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:34.684848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:34.685922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:34.686422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:34.686889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:34.699948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:34.702538 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:08:34.725243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:34.727190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:34.729072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:08:34.729072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:08:34.729072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:08:34.729072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:08:34.729072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:08:34.729072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:08:34.729072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:34.729072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:34.731658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 608/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:34.735381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:34.869929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e982-572a-4017-a59e-7d2fe1f91c48 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:34.897089 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:34.913439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b64846-2883-4a63-bf8a-a2ed89874e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:34.914411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b64846-2883-4a63-bf8a-a2ed89874e95 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:34.914886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b64846-2883-4a63-bf8a-a2ed89874e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:34.915375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b64846-2883-4a63-bf8a-a2ed89874e95 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:34.916663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b64846-2883-4a63-bf8a-a2ed89874e95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:34.917212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b64846-2883-4a63-bf8a-a2ed89874e95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:34.919849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b64846-2883-4a63-bf8a-a2ed89874e95 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:34.940741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 610/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:34.948072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:35.081503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7de36-098b-4baa-a863-259a89e494ef None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:35.120563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:35.129967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafeebcd-9e72-474b-9460-48f680bb53aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:35.131142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeebcd-9e72-474b-9460-48f680bb53aa None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:35.131586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeebcd-9e72-474b-9460-48f680bb53aa None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:35.132023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeebcd-9e72-474b-9460-48f680bb53aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:35.132800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafeebcd-9e72-474b-9460-48f680bb53aa None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=56361f445d894e73bae29764ae8b2b5d) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:35.133114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafeebcd-9e72-474b-9460-48f680bb53aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:35.135353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafeebcd-9e72-474b-9460-48f680bb53aa None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:35.153863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 609/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:08:34 2022] GET /identity/v3/role_assignments?user.id=56361f445d894e73bae29764ae8b2b5d&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:37.157189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3502-773f-4ada-99b8-dad89b816d24 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:37.157711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3502-773f-4ada-99b8-dad89b816d24 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:37.157711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3502-773f-4ada-99b8-dad89b816d24 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:37.159136 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 611/1220] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:08:37.166360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678db95-802e-4412-b50e-e5c51972565e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:37.166950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678db95-802e-4412-b50e-e5c51972565e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:37.166950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678db95-802e-4412-b50e-e5c51972565e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:37.186090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b678db95-802e-4412-b50e-e5c51972565e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:08:37.187360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b678db95-802e-4412-b50e-e5c51972565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:08:37.274262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 610/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:08:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:08:37.583313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf3502-773f-4ada-99b8-dad89b816d24 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:37.759127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf3502-773f-4ada-99b8-dad89b816d24 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:37.815324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca23da4-4efd-4d81-a9a5-48f38fff27b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:37.816295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca23da4-4efd-4d81-a9a5-48f38fff27b8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:37.816784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca23da4-4efd-4d81-a9a5-48f38fff27b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:37.817340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca23da4-4efd-4d81-a9a5-48f38fff27b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:37.838050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca23da4-4efd-4d81-a9a5-48f38fff27b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6ca23da4-4efd-4d81-a9a5-48f38fff27b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:08:37.848124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:37.849294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:08:37.850275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:37.851016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 612/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:08:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:37.857530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678db95-802e-4412-b50e-e5c51972565e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:38.016000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678db95-802e-4412-b50e-e5c51972565e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:38.050082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:38.065731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:38.066441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:38.066724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:38.067087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:38.067824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:38.068130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:38.070446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:38.080858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 611/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:08:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:08:38.088432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca23da4-4efd-4d81-a9a5-48f38fff27b8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:38.172549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:38.268568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca23da4-4efd-4d81-a9a5-48f38fff27b8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:38.325763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:38.326616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:38.326965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:38.327444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:38.343033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:38.344636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:38.362960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:38.364952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:38.366807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:38.366807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:38.366807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:38.366807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 613/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:38.373557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:38.519000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3f28b-404c-4393-b9eb-91e910215f60 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:38.569114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:38.570034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:38.570311 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:38.570572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:38.571270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:38.571563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:38.572979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:38.587153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 612/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:38.593317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:38.706663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:38.716173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568619f6-03d3-4785-a19d-c87406b6fa75 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:38.761750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:38.762448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:38.762726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:38.762996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:38.772791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:38.774700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:08:38.788229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:38.790198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:38.792374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:38.794067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 614/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:08:38.796072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:38.879918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:38.954111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0f327-4eb3-4db0-a727-6ddf75f0b8db None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:39.002849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:39.003796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:39.004147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:39.004574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:39.005639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:39.006087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:39.008032 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:39.033919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 613/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 31 12:08:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:39.042931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:39.203004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd05c99-3247-42cb-bacd-15771f9f2f66 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:39.258361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:39.259186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:39.259599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:39.260002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:39.277920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:39.280584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:39.304325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:39.306462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:39.308754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:39.308754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:39.308754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:39.308754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 615/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:08:39.313463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:39.467091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca597a-386d-4312-b3e4-6acd61c96515 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:39.470976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:39.525243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:39.525243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:39.525243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:39.525243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:39.525964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:39.525964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:39.526271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:39.542625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 614/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:39.550310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:39.564922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:39.721762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd2766-a09d-4080-86dc-0f4e03148e60 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:39.767787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:39.768488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:39.768880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:39.769127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:39.779591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:39.781843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:08:39.799349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:39.801367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:39.803605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 616/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:39.810577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:39.974024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0b5bf-6b94-4454-a1ef-e985b7f16e23 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:40.033650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b72b50-7d7b-4074-9294-54f5eb9d16de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:40.034416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b72b50-7d7b-4074-9294-54f5eb9d16de None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:40.034828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b72b50-7d7b-4074-9294-54f5eb9d16de None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:40.035226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b72b50-7d7b-4074-9294-54f5eb9d16de None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:40.036106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b72b50-7d7b-4074-9294-54f5eb9d16de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:40.036531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b72b50-7d7b-4074-9294-54f5eb9d16de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:40.042286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b72b50-7d7b-4074-9294-54f5eb9d16de None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:40.061978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 615/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:08:40.071474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:40.153772 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:40.194778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232f748-ae83-493b-b714-f5e709ed20d1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:40.240044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca30860-60eb-43a1-816d-8e1023c89da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:40.241128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30860-60eb-43a1-816d-8e1023c89da3 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:40.241489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30860-60eb-43a1-816d-8e1023c89da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:40.242052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30860-60eb-43a1-816d-8e1023c89da3 None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/56361f445d894e73bae29764ae8b2b5d/roles/aafb8817e1d640eb961fa58db7e05111` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:40.261726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca30860-60eb-43a1-816d-8e1023c89da3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=56361f445d894e73bae29764ae8b2b5d, role_id=aafb8817e1d640eb961fa58db7e05111)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:40.263762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca30860-60eb-43a1-816d-8e1023c89da3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:40.285901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 617/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:08:40 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/56361f445d894e73bae29764ae8b2b5d/roles/aafb8817e1d640eb961fa58db7e05111 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:08:42.421931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb05da7-f920-4528-a705-c4ad7293e118 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:42.423275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb05da7-f920-4528-a705-c4ad7293e118 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:42.423275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb05da7-f920-4528-a705-c4ad7293e118 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:42.424070 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 616/1233] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:08:42.431047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091a614-65f8-4263-9574-37268c802022 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:42.431646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091a614-65f8-4263-9574-37268c802022 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:42.431646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091a614-65f8-4263-9574-37268c802022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:42.450591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f091a614-65f8-4263-9574-37268c802022 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:08:42.451880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f091a614-65f8-4263-9574-37268c802022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:08:42.536520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 618/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:08:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:08:42.826175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb05da7-f920-4528-a705-c4ad7293e118 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:42.856311 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:42.968149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb05da7-f920-4528-a705-c4ad7293e118 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:43.011948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951bf1f-7cf6-4040-9f82-6434d8085f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:43.016920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951bf1f-7cf6-4040-9f82-6434d8085f89 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:43.017191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951bf1f-7cf6-4040-9f82-6434d8085f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:43.017467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951bf1f-7cf6-4040-9f82-6434d8085f89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:43.030013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951bf1f-7cf6-4040-9f82-6434d8085f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-5951bf1f-7cf6-4040-9f82-6434d8085f89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:08:43.034906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:43.036455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:43.037716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 617/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 31 12:08:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:43.045108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091a614-65f8-4263-9574-37268c802022 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:43.097115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:43.166345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091a614-65f8-4263-9574-37268c802022 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:43.209471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:43.210331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:43.210604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:43.210910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:43.211687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:43.212015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:43.213411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:43.223202 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 619/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 31 12:08:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:43.233367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951bf1f-7cf6-4040-9f82-6434d8085f89 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:43.351501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:43.371060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951bf1f-7cf6-4040-9f82-6434d8085f89 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:43.413733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:43.414370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:43.414649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:43.414924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:43.426421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:43.428134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:43.448149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:43.449597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:43.450817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:43.450817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:43.450817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:43.450817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 618/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:43.458604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:43.589720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3128f6-1ada-4c36-b2c9-ae123e8281a4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:43.602281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:43.633142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:43.633759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:43.634036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:43.634314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:43.634941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:43.635272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:43.636648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:43.659750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 620/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:43.666457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:43.795766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac39e2-4cef-4e01-8154-b283b41714f1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:43.836321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:43.836977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:43.837280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:43.837544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:43.840916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:43.847810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:43.849623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:08:43.862810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:43.864301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:43.865596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 619/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:43.872066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:44.000554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57198d5-3f65-499e-8f89-5f1cb5f26dde None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:44.062780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:44.063441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:44.063716 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:44.063982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:44.064733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:44.065031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:44.066474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:44.086182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 621/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 31 12:08:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:08:44.092102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:44.212042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbee667-1bef-44c0-8841-b6fce938d8f7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:44.257554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cacdaf-c962-42da-8f17-669cb0bfaeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:44.258178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cacdaf-c962-42da-8f17-669cb0bfaeef None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:44.258460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cacdaf-c962-42da-8f17-669cb0bfaeef None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:44.258726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cacdaf-c962-42da-8f17-669cb0bfaeef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:44.348220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:44.359043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cacdaf-c962-42da-8f17-669cb0bfaeef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:44.361721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cacdaf-c962-42da-8f17-669cb0bfaeef None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:44.503997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 620/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:08:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:08:44.509942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:44.543951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:44.692534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ee3e7-0de6-4e0b-8af4-518522c03654 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:44.742954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:44.744020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:44.744394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:44.744818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:44.758528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:44.759899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:44.784652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:44.785868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:44.787129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:44.787129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:44.787129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:44.787129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 622/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:08:44.794040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cacdaf-c962-42da-8f17-669cb0bfaeef None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:44.935567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:44.943842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cacdaf-c962-42da-8f17-669cb0bfaeef None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:44.997822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:44.998638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:44.999101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:44.999532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:45.000482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:45.000998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:45.003066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:45.020463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 621/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:45.027645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:45.150803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:45.185401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8e932-1f94-4dcd-b542-f8e8481aaf0c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:45.241632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:45.242367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:45.242645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:45.242911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:45.251103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:45.252768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:08:45.269859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:45.278232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:45.279327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 623/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:08:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:08:45.280386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:45.430437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b93db4-4c96-4142-9687-4f4745e929f7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:45.476205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712b1df-12e6-4ea7-89d4-b9608a41d86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:45.477315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b1df-12e6-4ea7-89d4-b9608a41d86c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:45.477926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b1df-12e6-4ea7-89d4-b9608a41d86c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:45.478363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b1df-12e6-4ea7-89d4-b9608a41d86c None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:45.479416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712b1df-12e6-4ea7-89d4-b9608a41d86c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:45.479986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712b1df-12e6-4ea7-89d4-b9608a41d86c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:45.481799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712b1df-12e6-4ea7-89d4-b9608a41d86c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:45.500110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 622/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:08:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:45.508409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:45.694118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e19d95-3328-48e4-b1a1-438a698f8751 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:45.751679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1582e-11fb-4d1e-96d7-0a37c95ebb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:45.752408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1582e-11fb-4d1e-96d7-0a37c95ebb3f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:45.752877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1582e-11fb-4d1e-96d7-0a37c95ebb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:45.753308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1582e-11fb-4d1e-96d7-0a37c95ebb3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:45.754535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1582e-11fb-4d1e-96d7-0a37c95ebb3f None admin] RBAC: Adding query filter params (role.id=aafb8817e1d640eb961fa58db7e05111, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=56361f445d894e73bae29764ae8b2b5d) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:45.755021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1582e-11fb-4d1e-96d7-0a37c95ebb3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:45.758199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1582e-11fb-4d1e-96d7-0a37c95ebb3f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:45.768000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 624/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:08:45 2022] GET /identity/v3/role_assignments?user.id=56361f445d894e73bae29764ae8b2b5d&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=aafb8817e1d640eb961fa58db7e05111 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:08:48.184450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89be5c-8d8b-48ff-b2eb-bc01bcff296a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:48.185280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89be5c-8d8b-48ff-b2eb-bc01bcff296a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:48.185468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89be5c-8d8b-48ff-b2eb-bc01bcff296a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:48.187353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 623/1247] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:08:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:08:48.194390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d1ff-6c61-424a-845f-5a0285a0b5cb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:48.195311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d1ff-6c61-424a-845f-5a0285a0b5cb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:48.195422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d1ff-6c61-424a-845f-5a0285a0b5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:48.203075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:48.218724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f4a6d1ff-6c61-424a-845f-5a0285a0b5cb None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:08:48.220422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f4a6d1ff-6c61-424a-845f-5a0285a0b5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:08:48.315224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 625/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:08:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:08:48.597801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89be5c-8d8b-48ff-b2eb-bc01bcff296a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:48.625225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:48.738804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89be5c-8d8b-48ff-b2eb-bc01bcff296a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:48.792241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf746d7-91f1-492c-a067-31edc6bc2bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:48.793019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf746d7-91f1-492c-a067-31edc6bc2bee None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:48.793345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf746d7-91f1-492c-a067-31edc6bc2bee None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:48.793700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf746d7-91f1-492c-a067-31edc6bc2bee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:48.809576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf746d7-91f1-492c-a067-31edc6bc2bee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-caf746d7-91f1-492c-a067-31edc6bc2bee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:08:48.817831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:48.820220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:48.822130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 624/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:08:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:48.826376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d1ff-6c61-424a-845f-5a0285a0b5cb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:48.967959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d1ff-6c61-424a-845f-5a0285a0b5cb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:49.012768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:49.019974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:49.020941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:49.021102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:49.021350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:49.022869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:49.022869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:49.024292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:49.035295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 626/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:08:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:08:49.042688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf746d7-91f1-492c-a067-31edc6bc2bee None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:49.156389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:49.194101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf746d7-91f1-492c-a067-31edc6bc2bee None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:49.245043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:49.246099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:49.246590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:49.247066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:49.263206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:49.265373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:49.294160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:49.295460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:49.296893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:49.296893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:49.296893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:49.296893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 625/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:49.305958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:49.460151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede2188-de87-42d3-82f6-d1266d60440d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:49.510668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:49.511462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:49.511932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:49.512593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:49.513860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:49.514303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:49.516541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:49.535352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 627/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:49.544067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:49.708407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240cd12-3697-46fb-a034-90c5caee6a30 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:49.752990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:49.767629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:49.768640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:49.769184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:49.769669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:49.784892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:49.787212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:08:49.808878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:49.810254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:49.811319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 626/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:08:49.818304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:49.915631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:49.988640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eba505-0506-46b9-8191-c7bca8abaa7d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:50.038183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:50.039070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:50.039579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:50.039988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:50.040911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:50.041336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:50.042944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:50.067884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 628/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 31 12:08:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:50.074858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:50.226113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80daf476-30ba-4441-b118-f2f2fab830d3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:50.272063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806af92f-fc82-4ea6-97dc-bd2b222b82b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:50.273066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806af92f-fc82-4ea6-97dc-bd2b222b82b6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:50.273345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806af92f-fc82-4ea6-97dc-bd2b222b82b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:50.273657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806af92f-fc82-4ea6-97dc-bd2b222b82b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:50.362747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:50.379470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806af92f-fc82-4ea6-97dc-bd2b222b82b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:50.382210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806af92f-fc82-4ea6-97dc-bd2b222b82b6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:50.528708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 627/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:08:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:08:50.535395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:50.682437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9fffc-80ef-4cb6-82b8-e8c1e83a649a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:50.702242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:50.740795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:50.742079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:50.742802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:50.743343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:50.768563 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:50.771200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:50.801588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:50.804086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:50.806209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:50.806209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:50.806209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:50.806209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 629/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:50.812688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806af92f-fc82-4ea6-97dc-bd2b222b82b6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:50.962809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:50.980767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806af92f-fc82-4ea6-97dc-bd2b222b82b6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:51.036278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:51.037498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:51.038268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:51.038649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:51.039834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:51.040330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:51.042534 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:51.063159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 628/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:08:51.070545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:51.222541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f615d-d222-4b66-ae67-7fd05c2a5a9d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:51.279367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:51.280559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:51.281147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:51.281583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:51.293173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:51.295335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:08:51.312271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:51.313527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:51.314525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 630/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:51.320854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:51.440893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30ab6a-b797-45bb-8ea3-ca1e10b055b8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:51.494249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653c9ac-3b51-418d-9ec8-cc1a8010fb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:51.494784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c9ac-3b51-418d-9ec8-cc1a8010fb4b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:51.495065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c9ac-3b51-418d-9ec8-cc1a8010fb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:51.495385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c9ac-3b51-418d-9ec8-cc1a8010fb4b None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:51.496082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653c9ac-3b51-418d-9ec8-cc1a8010fb4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:51.496476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653c9ac-3b51-418d-9ec8-cc1a8010fb4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:51.498011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653c9ac-3b51-418d-9ec8-cc1a8010fb4b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:51.511667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 629/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:51.522819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:51.555556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:51.662620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffde473-c381-446f-99a4-43e153086066 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:51.721634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea75039-c106-4160-8d7d-34eed238cdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:51.722554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75039-c106-4160-8d7d-34eed238cdad None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:51.722914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75039-c106-4160-8d7d-34eed238cdad None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:51.723404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75039-c106-4160-8d7d-34eed238cdad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:51.724406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea75039-c106-4160-8d7d-34eed238cdad None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=56361f445d894e73bae29764ae8b2b5d) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:51.724835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea75039-c106-4160-8d7d-34eed238cdad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:51.727095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea75039-c106-4160-8d7d-34eed238cdad None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:51.735127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 631/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:08:51 2022] GET /identity/v3/role_assignments?user.id=56361f445d894e73bae29764ae8b2b5d&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:53.737433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7234eb3-03ce-4d74-8cd2-6251d41c82aa None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:53.737979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7234eb3-03ce-4d74-8cd2-6251d41c82aa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:53.737979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7234eb3-03ce-4d74-8cd2-6251d41c82aa None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:53.739376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 630/1261] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:08:53.745566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f82f9-37da-4e20-933a-d04be6c530c7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:53.746178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f82f9-37da-4e20-933a-d04be6c530c7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:53.746178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f82f9-37da-4e20-933a-d04be6c530c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:53.774364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c18f82f9-37da-4e20-933a-d04be6c530c7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:08:53.776484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c18f82f9-37da-4e20-933a-d04be6c530c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:08:53.887504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 632/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:08:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:08:54.162934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7234eb3-03ce-4d74-8cd2-6251d41c82aa None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:54.174622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:54.279780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7234eb3-03ce-4d74-8cd2-6251d41c82aa None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:54.332936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5764d9-34bd-43aa-aab6-8b9d8843b06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:54.333571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5764d9-34bd-43aa-aab6-8b9d8843b06d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:54.333894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5764d9-34bd-43aa-aab6-8b9d8843b06d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:54.334235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5764d9-34bd-43aa-aab6-8b9d8843b06d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:54.346335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5764d9-34bd-43aa-aab6-8b9d8843b06d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4b5764d9-34bd-43aa-aab6-8b9d8843b06d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:08:54.352218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:54.353841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:54.354740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 631/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:08:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:54.359657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f82f9-37da-4e20-933a-d04be6c530c7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:54.380126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:54.467633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f82f9-37da-4e20-933a-d04be6c530c7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:54.503822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:54.504437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:54.504706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:54.504968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:54.505634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:54.506221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:54.507816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:54.516748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 633/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:54.525851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5764d9-34bd-43aa-aab6-8b9d8843b06d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:54.600444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:54.636816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5764d9-34bd-43aa-aab6-8b9d8843b06d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:54.673468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:54.674107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:54.674384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:54.674676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:54.687580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:54.689057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:54.705028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:54.706597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:54.708647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:54.708647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:54.708647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:54.708647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 632/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:54.714218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:54.801732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:54.824930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dca672-e5f5-4b35-91ff-ee3ebbdc3cbe None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:54.860824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:54.861439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:54.861710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:54.861975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:54.862627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:54.862922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:54.864359 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:54.880160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 634/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:54.886800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:55.014576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84e02f-5598-4ec4-89e3-df3d504d51c0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:55.039669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:55.057514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:55.058065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:55.058338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:55.058603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:55.067218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:55.068794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:08:55.081482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:55.082812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:55.084363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 633/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 12:08:55.092297 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:55.225110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a4bca-6d04-44e0-b761-540d1584b622 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:55.282374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:55.283075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:55.283388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:55.283728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:55.284487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:55.284769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:55.286415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:55.298376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:55.314389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 635/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 31 12:08:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:08:55.321736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:55.440003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f80c38-648c-4229-accf-de3563b7a9f1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:55.481181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:55.481827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:55.482104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:55.482373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:55.496552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:55.498084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:55.515178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:55.516916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:55.517903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:55.517903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:55.517903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:55.517903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 634/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:55.523116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:55.645377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b920f-d555-4e89-844d-005d952452c4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:55.683892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:55.684787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:55.685194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:55.685589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:55.686568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:55.687002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:55.689191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:55.706962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 636/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:55.713915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:55.754998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:55.832975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b329b-26a8-4bc1-9544-c9c2042b9577 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:55.877425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:55.878160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:55.878489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:55.878809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:55.888074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:55.889915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:08:55.904668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:55.906248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:08:55.907970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 635/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:55.913683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:55.982690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:56.032438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a7ce0-047a-4f87-acdd-f3455f8fa652 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:56.068693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084b899-1195-4af7-9b01-8d0abc700f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:56.069441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b899-1195-4af7-9b01-8d0abc700f65 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:56.069830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b899-1195-4af7-9b01-8d0abc700f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:56.070097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b899-1195-4af7-9b01-8d0abc700f65 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:56.070787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084b899-1195-4af7-9b01-8d0abc700f65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:56.071088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084b899-1195-4af7-9b01-8d0abc700f65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:56.072585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084b899-1195-4af7-9b01-8d0abc700f65 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:56.085567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 637/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:08:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:56.092113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:56.190843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:56.200910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb346c-a6da-4dee-b47e-f45968abdae5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:56.236576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38312d2-2e2f-4981-b1f4-92f18032a7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:56.237295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38312d2-2e2f-4981-b1f4-92f18032a7ff None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:56.237615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38312d2-2e2f-4981-b1f4-92f18032a7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:56.237925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38312d2-2e2f-4981-b1f4-92f18032a7ff None admin] PATH_INFO: `/v3/projects/6c0eb399536f48f18191d97441cea577/users/56361f445d894e73bae29764ae8b2b5d/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:56.240128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38312d2-2e2f-4981-b1f4-92f18032a7ff None admin] RBAC: Authorizing `identity:create_grant(project_id=6c0eb399536f48f18191d97441cea577, user_id=56361f445d894e73bae29764ae8b2b5d, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:56.241673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38312d2-2e2f-4981-b1f4-92f18032a7ff None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:56.256274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 636/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 31 12:08:56 2022] PUT /identity/v3/projects/6c0eb399536f48f18191d97441cea577/users/56361f445d894e73bae29764ae8b2b5d/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:08:58.364806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd4ef0-0b00-490a-a2fa-4ae8dd6e12d9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:58.365471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd4ef0-0b00-490a-a2fa-4ae8dd6e12d9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:58.365642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd4ef0-0b00-490a-a2fa-4ae8dd6e12d9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:58.367070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 638/1274] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:08:58.376174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899f738-5efc-48b2-87bf-0c5028b37ea0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:58.376954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899f738-5efc-48b2-87bf-0c5028b37ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:58.377131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899f738-5efc-48b2-87bf-0c5028b37ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:58.407746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4899f738-5efc-48b2-87bf-0c5028b37ea0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:08:58.409787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4899f738-5efc-48b2-87bf-0c5028b37ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:08:58.527566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 637/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:08:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:08:58.897741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd4ef0-0b00-490a-a2fa-4ae8dd6e12d9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:59.014416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd4ef0-0b00-490a-a2fa-4ae8dd6e12d9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:59.033443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:59.051751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686f36c-ce6b-42bb-aed2-22d85b6c6977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:59.052392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f36c-ce6b-42bb-aed2-22d85b6c6977 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:59.052747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f36c-ce6b-42bb-aed2-22d85b6c6977 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:59.053055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f36c-ce6b-42bb-aed2-22d85b6c6977 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:59.064330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686f36c-ce6b-42bb-aed2-22d85b6c6977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d686f36c-ce6b-42bb-aed2-22d85b6c6977 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 12:08:59.069211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:59.070677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:59.071809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 639/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 31 12:08:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:59.076740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899f738-5efc-48b2-87bf-0c5028b37ea0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:59.182094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:59.204626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899f738-5efc-48b2-87bf-0c5028b37ea0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:59.242794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:59.243446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:59.243729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:59.243998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:59.244667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:59.244971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:59.246363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:59.255324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 638/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 31 12:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:59.261911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686f36c-ce6b-42bb-aed2-22d85b6c6977 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:59.400175 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686f36c-ce6b-42bb-aed2-22d85b6c6977 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:59.436643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:59.437179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:59.437448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:59.437719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:59.448497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:59.449961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:08:59.465679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:59.467220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:08:59.469141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:08:59.469141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:08:59.469141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:59.469141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 640/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:59.473953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:59.597731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78562659-be50-4751-9cbe-ff68e501d458 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:59.641257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:59.641891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:59.642165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:59.642432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:59.643085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:08:59.643425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:59.644893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:59.658818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 639/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:08:59.667803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:59.701628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:59.778770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36901b38-99d0-40d8-aac6-7bc305058171 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:08:59.815929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:08:59.816594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:08:59.816865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:08:59.817128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:08:59.825690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:08:59.827147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 12:08:59.839936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:08:59.841586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:08:59.842553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 641/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:08:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:08:59.851483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:08:59.855875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:08:59.981553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294cc7-1942-4272-be23-c6a1f2206783 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:00.023751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:00.024387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:00.024663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:00.024934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:00.025679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:00.025997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:00.027591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:00.049589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 640/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 31 12:08:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:09:00.056697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:00.169194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:00.181404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1262314-7734-4ab0-b791-590c9577c2f0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:00.220742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d156e-f0e6-4be2-ab98-3e4bf0ec30d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:00.221369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d156e-f0e6-4be2-ab98-3e4bf0ec30d7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:00.221647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d156e-f0e6-4be2-ab98-3e4bf0ec30d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:00.221899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d156e-f0e6-4be2-ab98-3e4bf0ec30d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:00.300071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d156e-f0e6-4be2-ab98-3e4bf0ec30d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:00.301431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d156e-f0e6-4be2-ab98-3e4bf0ec30d7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:00.398147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:00.409292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 642/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 31 12:09:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 12:09:00.415837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:00.485632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:00.529144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e3c9d-8704-4a08-ab2c-66f023835b62 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:00.567520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:00.568116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:00.568399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:00.569352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:00.580991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:00.582571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 12:09:00.601101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:09:00.603218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 12:09:00.605338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 12:09:00.605338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 12:09:00.605338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:09:00.605338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 641/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:09:00.610288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d156e-f0e6-4be2-ab98-3e4bf0ec30d7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:00.783651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d156e-f0e6-4be2-ab98-3e4bf0ec30d7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:00.824777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:00.825316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:00.825579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:00.825850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:00.826496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:00.826795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:00.828242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:00.841005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 643/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 31 12:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:09:00.847620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:00.996582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2d15b-838a-4a79-a18d-fcd1fa4e1d7f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:01.013988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:01.037468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:01.038222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:01.038504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:01.038778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:01.046250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:01.047909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 12:09:01.060398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:09:01.062109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:09:01.063889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 642/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 31 12:09:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:09:01.068296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:01.126171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:01.186870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17849b6-270d-4d27-aeb0-60c440a363d0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:01.225939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34288590-5b29-4b70-9f10-f38e10e65aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:01.226562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288590-5b29-4b70-9f10-f38e10e65aaa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:01.226842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288590-5b29-4b70-9f10-f38e10e65aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:01.227113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34288590-5b29-4b70-9f10-f38e10e65aaa None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:01.227853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34288590-5b29-4b70-9f10-f38e10e65aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:01.228158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34288590-5b29-4b70-9f10-f38e10e65aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:01.229608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34288590-5b29-4b70-9f10-f38e10e65aaa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:01.243346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 644/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 31 12:09:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:09:01.250004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:01.393286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43990d7-4edd-4b6a-a2b0-771b1e3239e2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:01.430115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debf533-1cf4-4fab-a974-8e84021b0b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:01.430728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf533-1cf4-4fab-a974-8e84021b0b00 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:01.431004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf533-1cf4-4fab-a974-8e84021b0b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:01.431282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf533-1cf4-4fab-a974-8e84021b0b00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:01.432034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debf533-1cf4-4fab-a974-8e84021b0b00 None admin] RBAC: Adding query filter params (role.id=ce6acb63b60a4f53a9ff9a32054b9b63, scope.project.id=6c0eb399536f48f18191d97441cea577, user.id=56361f445d894e73bae29764ae8b2b5d) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:01.432341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debf533-1cf4-4fab-a974-8e84021b0b00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:01.433787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debf533-1cf4-4fab-a974-8e84021b0b00 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:01.440886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 643/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 31 12:09:01 2022] GET /identity/v3/role_assignments?user.id=56361f445d894e73bae29764ae8b2b5d&scope.project.id=6c0eb399536f48f18191d97441cea577&role.id=ce6acb63b60a4f53a9ff9a32054b9b63 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:09:03.441031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244907bb-a28c-414b-880d-84e92355a55e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:03.441616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244907bb-a28c-414b-880d-84e92355a55e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:03.441616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244907bb-a28c-414b-880d-84e92355a55e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:03.442907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 645/1288] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:09:03.450048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3024ab6-ae01-429a-83f9-1eec6aa0d266 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:03.450651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3024ab6-ae01-429a-83f9-1eec6aa0d266 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:03.450974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3024ab6-ae01-429a-83f9-1eec6aa0d266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:03.480760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f3024ab6-ae01-429a-83f9-1eec6aa0d266 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:09:03.482710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f3024ab6-ae01-429a-83f9-1eec6aa0d266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:09:03.535909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:03.597458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 644/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:09:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:09:03.879580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244907bb-a28c-414b-880d-84e92355a55e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:03.993093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244907bb-a28c-414b-880d-84e92355a55e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:04.004871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:04.032033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:04.032729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:04.033054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:04.033370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:04.034002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:04.035503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:09:04.045485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:09:04.047217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:04.048921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 646/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:09:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:09:04.056363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3024ab6-ae01-429a-83f9-1eec6aa0d266 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:04.166009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3024ab6-ae01-429a-83f9-1eec6aa0d266 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:04.203444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0164e1-fd29-474e-9c76-5244243d2a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:04.204061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0164e1-fd29-474e-9c76-5244243d2a16 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:04.204339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0164e1-fd29-474e-9c76-5244243d2a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:04.204612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0164e1-fd29-474e-9c76-5244243d2a16 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:04.205239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0164e1-fd29-474e-9c76-5244243d2a16 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:04.205544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0164e1-fd29-474e-9c76-5244243d2a16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:04.206939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0164e1-fd29-474e-9c76-5244243d2a16 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:04.212431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 645/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 31 12:09:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:09:04.218584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:04.328448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58275a-ba35-4b06-b3bd-715991038e01 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:04.366865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a58fa-d12f-4695-bbbe-ed64ea40f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:04.367508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a58fa-d12f-4695-bbbe-ed64ea40f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:04.367787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a58fa-d12f-4695-bbbe-ed64ea40f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:04.368062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a58fa-d12f-4695-bbbe-ed64ea40f2e0 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:04.368707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a58fa-d12f-4695-bbbe-ed64ea40f2e0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:04.369005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a58fa-d12f-4695-bbbe-ed64ea40f2e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:04.370398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a58fa-d12f-4695-bbbe-ed64ea40f2e0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:04.375509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 647/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 31 12:09:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:09:06.303810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537491fb-b40a-4763-9b99-445efecc9eb6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:06.304386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537491fb-b40a-4763-9b99-445efecc9eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:06.304386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537491fb-b40a-4763-9b99-445efecc9eb6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:06.305840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 646/1293] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:09:06.312153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aae4d-b507-4a07-9a8d-6aab83ebf7a2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:06.312932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aae4d-b507-4a07-9a8d-6aab83ebf7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:06.312932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aae4d-b507-4a07-9a8d-6aab83ebf7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:06.336774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-154aae4d-b507-4a07-9a8d-6aab83ebf7a2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:09:06.337973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-154aae4d-b507-4a07-9a8d-6aab83ebf7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:09:06.372367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:06.414637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 648/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:09:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 12:09:06.688884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537491fb-b40a-4763-9b99-445efecc9eb6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:06.732187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:06.861104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537491fb-b40a-4763-9b99-445efecc9eb6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:06.915903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a8c2-3625-494b-b88b-0d7744adb0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:06.916522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a8c2-3625-494b-b88b-0d7744adb0f0 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:06.916797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a8c2-3625-494b-b88b-0d7744adb0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:06.917060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a8c2-3625-494b-b88b-0d7744adb0f0 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:06.917709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503a8c2-3625-494b-b88b-0d7744adb0f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:06.919081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503a8c2-3625-494b-b88b-0d7744adb0f0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:06.935810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 647/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 31 12:09:06 2022] POST /identity/v3/services => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:09:08.952889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d5307-435a-4db0-84db-56b3b5d516f4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:08.953467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d5307-435a-4db0-84db-56b3b5d516f4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:08.953467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d5307-435a-4db0-84db-56b3b5d516f4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:08.954756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 649/1296] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:09:08.960402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b333458-af51-4ccf-a8be-424b73e6a143 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:08.960889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b333458-af51-4ccf-a8be-424b73e6a143 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:08.961009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b333458-af51-4ccf-a8be-424b73e6a143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:08.981007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6b333458-af51-4ccf-a8be-424b73e6a143 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:09:08.982248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6b333458-af51-4ccf-a8be-424b73e6a143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:09:09.062110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 648/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:09:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:09:09.327115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d5307-435a-4db0-84db-56b3b5d516f4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:09.442390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d5307-435a-4db0-84db-56b3b5d516f4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:09.479051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:09.479708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:09.479988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:09.480262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:09.480846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:09.482218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:09:09.491825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:09:09.493691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:09.495791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 650/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:09:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:09:09.500970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b333458-af51-4ccf-a8be-424b73e6a143 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:09.520707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:09.613380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b333458-af51-4ccf-a8be-424b73e6a143 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:09.653121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:09.653785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:09.654099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:09.654405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:09.655083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:09.655458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:09.656891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:09.662569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 649/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 31 12:09:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:09:09.669056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:09.715522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:09.790096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcbabf-5401-4ac1-8084-3ae1a8141380 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:09.825206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b497c-f0f1-4de9-99fc-cc1f3fc60950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:09.825796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b497c-f0f1-4de9-99fc-cc1f3fc60950 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:09.826067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b497c-f0f1-4de9-99fc-cc1f3fc60950 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:09.826333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b497c-f0f1-4de9-99fc-cc1f3fc60950 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:09.827018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b497c-f0f1-4de9-99fc-cc1f3fc60950 None admin] RBAC: Adding query filter params (interface=public, service_id=aee19719bc6846d8b6c7924107d89e79, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:09.827342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b497c-f0f1-4de9-99fc-cc1f3fc60950 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:09.828701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b497c-f0f1-4de9-99fc-cc1f3fc60950 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:09.834254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 651/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 31 12:09:09 2022] GET /identity/v3/endpoints?service_id=aee19719bc6846d8b6c7924107d89e79&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:09:09.840407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:09.945424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:09.975093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6d8d4-1bd1-42fb-992c-47e9f940ab5b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:10.021960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c41ab5-d804-4639-ac60-c561ae2c3475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:10.022585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c41ab5-d804-4639-ac60-c561ae2c3475 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:10.022858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c41ab5-d804-4639-ac60-c561ae2c3475 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:10.023124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c41ab5-d804-4639-ac60-c561ae2c3475 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:10.023767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c41ab5-d804-4639-ac60-c561ae2c3475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:10.025160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c41ab5-d804-4639-ac60-c561ae2c3475 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:10.032838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 650/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 31 12:09:09 2022] GET /identity/v3/services => generated 2191 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:09:12.254113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b594d-e109-4ecf-890d-e7107386ecb9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:12.254730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b594d-e109-4ecf-890d-e7107386ecb9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:12.254730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b594d-e109-4ecf-890d-e7107386ecb9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:12.256085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 652/1302] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:09:12.261747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4191-c893-40b1-ad3f-711654df474a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:12.262252 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4191-c893-40b1-ad3f-711654df474a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:12.262378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4191-c893-40b1-ad3f-711654df474a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:12.285598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5cca4191-c893-40b1-ad3f-711654df474a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:09:12.288145 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5cca4191-c893-40b1-ad3f-711654df474a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:09:12.397392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 651/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:09:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 12:09:12.665765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b594d-e109-4ecf-890d-e7107386ecb9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:12.788430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:12.796197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b594d-e109-4ecf-890d-e7107386ecb9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:12.832248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:12.832884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:12.833155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:12.833507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:12.834095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:12.835501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:09:12.844656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:09:12.846664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: ERROR keystone.server.flask.application Aug 31 12:09:12.847700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 653/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 31 12:09:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 12:09:12.855470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca4191-c893-40b1-ad3f-711654df474a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:12.945322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:12.972264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca4191-c893-40b1-ad3f-711654df474a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:13.012242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38893e-ad69-47a2-88dd-e7a4cf7c7527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:13.012865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38893e-ad69-47a2-88dd-e7a4cf7c7527 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:13.013308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38893e-ad69-47a2-88dd-e7a4cf7c7527 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:13.013721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38893e-ad69-47a2-88dd-e7a4cf7c7527 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:13.014503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38893e-ad69-47a2-88dd-e7a4cf7c7527 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:09:13.014810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38893e-ad69-47a2-88dd-e7a4cf7c7527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:13.016245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38893e-ad69-47a2-88dd-e7a4cf7c7527 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:13.022509 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 652/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 31 12:09:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:09:13.032454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:09:13.178794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36040a59-38b8-45a4-9f42-2952727b53a6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:09:13.226585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd85e7-7a19-42f1-8cb4-92fd08854c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:09:13.227277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd85e7-7a19-42f1-8cb4-92fd08854c57 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:09:13.227637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd85e7-7a19-42f1-8cb4-92fd08854c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:09:13.227968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd85e7-7a19-42f1-8cb4-92fd08854c57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:09:13.228733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd85e7-7a19-42f1-8cb4-92fd08854c57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:09:13.230215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd85e7-7a19-42f1-8cb4-92fd08854c57 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:09:13.254985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 654/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:09:13 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:09:23.018800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=30 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:09:23.249844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:21.752687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805ac71-5ce2-48e4-9159-9fc210bed955 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:21.753355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805ac71-5ce2-48e4-9159-9fc210bed955 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:21.753355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805ac71-5ce2-48e4-9159-9fc210bed955 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:21.758615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 653/1307] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:21.760161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71555f07-10da-4721-8ade-b3f20fabee2a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:21.760635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71555f07-10da-4721-8ade-b3f20fabee2a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:21.761803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71555f07-10da-4721-8ade-b3f20fabee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:21.942500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-71555f07-10da-4721-8ade-b3f20fabee2a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:21.943952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-71555f07-10da-4721-8ade-b3f20fabee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:22.114422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 655/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:22.407505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99159c2-9c2e-49ac-9d47-ed7c590278b0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:22.407759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99159c2-9c2e-49ac-9d47-ed7c590278b0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:22.408341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99159c2-9c2e-49ac-9d47-ed7c590278b0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:22.409788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 654/1309] 10.222.0.27 () {54 vars in 965 bytes} [Wed Aug 31 12:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:22.416185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6fcac-a372-47c6-8dd0-1eac05977f52 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:22.416504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6fcac-a372-47c6-8dd0-1eac05977f52 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:22.416880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6fcac-a372-47c6-8dd0-1eac05977f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:22.500415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eee6fcac-a372-47c6-8dd0-1eac05977f52 None None] MFA Rules not processed for user `ba8c7348ad2e457e80e1fdd2d93aacc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:22.501650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eee6fcac-a372-47c6-8dd0-1eac05977f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:22.524919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:22.586463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 656/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Wed Aug 31 12:10:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:22.595776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99159c2-9c2e-49ac-9d47-ed7c590278b0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:22.597718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c99159c2-9c2e-49ac-9d47-ed7c590278b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:22.717791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99159c2-9c2e-49ac-9d47-ed7c590278b0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:22.796900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a316d-b169-4a58-9acf-322c3195ed4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:22.798419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a316d-b169-4a58-9acf-322c3195ed4b service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:22.798896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a316d-b169-4a58-9acf-322c3195ed4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:22.799369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a316d-b169-4a58-9acf-322c3195ed4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:22.801258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a90a316d-b169-4a58-9acf-322c3195ed4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:22.869994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:22.947532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a316d-b169-4a58-9acf-322c3195ed4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:22.949145 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a316d-b169-4a58-9acf-322c3195ed4b service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:22.976966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 655/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:25.303740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a72ff-1103-4368-ac59-805038d8ae12 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:25.304330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a72ff-1103-4368-ac59-805038d8ae12 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:25.304330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a72ff-1103-4368-ac59-805038d8ae12 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:25.305631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 657/1312] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:25.311399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9dbf9-1c40-40f5-93c8-8917cd75ce41 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:25.311891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9dbf9-1c40-40f5-93c8-8917cd75ce41 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:25.311891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9dbf9-1c40-40f5-93c8-8917cd75ce41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:25.386185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2af9dbf9-1c40-40f5-93c8-8917cd75ce41 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:25.387333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2af9dbf9-1c40-40f5-93c8-8917cd75ce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:25.485767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 656/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:25.759629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a72ff-1103-4368-ac59-805038d8ae12 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:25.761686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-180a72ff-1103-4368-ac59-805038d8ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:25.869162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a72ff-1103-4368-ac59-805038d8ae12 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:25.943751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:25.946584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def033e7-dfd7-4743-886d-be85b7120f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:25.947263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def033e7-dfd7-4743-886d-be85b7120f5e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:25.947538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def033e7-dfd7-4743-886d-be85b7120f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:25.947812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def033e7-dfd7-4743-886d-be85b7120f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:25.949848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-def033e7-dfd7-4743-886d-be85b7120f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:26.060027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def033e7-dfd7-4743-886d-be85b7120f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:26.061380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def033e7-dfd7-4743-886d-be85b7120f5e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:26.081670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 658/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:28.357954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748f49-06e5-4d7e-98be-d08d4cbfe4ae None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:28.358518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748f49-06e5-4d7e-98be-d08d4cbfe4ae None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:28.358518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748f49-06e5-4d7e-98be-d08d4cbfe4ae None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:28.359857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 657/1315] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:28.365114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19beb25-89f4-4951-950d-8bd998a79ef1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:28.365567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19beb25-89f4-4951-950d-8bd998a79ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:28.365661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19beb25-89f4-4951-950d-8bd998a79ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:28.433032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d19beb25-89f4-4951-950d-8bd998a79ef1 None None] MFA Rules not processed for user `16aab776604b46f6b9f706d4d684a6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:28.434235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d19beb25-89f4-4951-950d-8bd998a79ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:28.488639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:28.535305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 659/1316] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:28.818828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab748f49-06e5-4d7e-98be-d08d4cbfe4ae None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:28.820426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ab748f49-06e5-4d7e-98be-d08d4cbfe4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:28.830683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:28.934797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab748f49-06e5-4d7e-98be-d08d4cbfe4ae None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:29.009980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63172429-b1be-49a3-ae44-13ce37da62aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:29.010592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63172429-b1be-49a3-ae44-13ce37da62aa service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:29.010867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63172429-b1be-49a3-ae44-13ce37da62aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:29.011134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63172429-b1be-49a3-ae44-13ce37da62aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:29.012801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-63172429-b1be-49a3-ae44-13ce37da62aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:29.079752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:29.131099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63172429-b1be-49a3-ae44-13ce37da62aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:29.132574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63172429-b1be-49a3-ae44-13ce37da62aa service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:29.151470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 658/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:31.643200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f718d4-d41f-484f-a5e9-b8a825623728 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:31.643820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f718d4-d41f-484f-a5e9-b8a825623728 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:31.643820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f718d4-d41f-484f-a5e9-b8a825623728 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:31.645541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 660/1318] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:31.651150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f8a03-9815-48ef-a67e-17fb935ae71f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:31.651602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f8a03-9815-48ef-a67e-17fb935ae71f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:31.651742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f8a03-9815-48ef-a67e-17fb935ae71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:31.726876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cc5f8a03-9815-48ef-a67e-17fb935ae71f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:31.728121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cc5f8a03-9815-48ef-a67e-17fb935ae71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:31.829416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 659/1319] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:32.116091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f718d4-d41f-484f-a5e9-b8a825623728 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:32.117640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-01f718d4-d41f-484f-a5e9-b8a825623728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:32.226483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f718d4-d41f-484f-a5e9-b8a825623728 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:32.301706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9272a-6acf-4963-bf2d-5dce0426ad94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:32.302308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9272a-6acf-4963-bf2d-5dce0426ad94 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:32.302580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9272a-6acf-4963-bf2d-5dce0426ad94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:32.302848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9272a-6acf-4963-bf2d-5dce0426ad94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:32.304487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8b9272a-6acf-4963-bf2d-5dce0426ad94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:32.313067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:32.415501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9272a-6acf-4963-bf2d-5dce0426ad94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:32.416936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9272a-6acf-4963-bf2d-5dce0426ad94 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:32.435084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 661/1320] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:35.020998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dffbb5-c94f-424b-b464-00feaea46285 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:35.021551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dffbb5-c94f-424b-b464-00feaea46285 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:35.021551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dffbb5-c94f-424b-b464-00feaea46285 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:35.022901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 660/1321] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:35.028546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c03e9-02fa-4d0e-b789-0c3b811b5ab2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:35.029084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c03e9-02fa-4d0e-b789-0c3b811b5ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:35.029084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c03e9-02fa-4d0e-b789-0c3b811b5ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:35.105583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fe6c03e9-02fa-4d0e-b789-0c3b811b5ab2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:35.106838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fe6c03e9-02fa-4d0e-b789-0c3b811b5ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:35.171191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:35.212757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 662/1322] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:35.493851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dffbb5-c94f-424b-b464-00feaea46285 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:35.495523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-97dffbb5-c94f-424b-b464-00feaea46285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:35.516799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:35.619746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dffbb5-c94f-424b-b464-00feaea46285 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:35.696543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be425e74-f028-4fac-be5a-08336161c76b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:35.697156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be425e74-f028-4fac-be5a-08336161c76b service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:35.697431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be425e74-f028-4fac-be5a-08336161c76b service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:35.697698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be425e74-f028-4fac-be5a-08336161c76b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:35.699336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-be425e74-f028-4fac-be5a-08336161c76b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:35.778466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:35.823498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be425e74-f028-4fac-be5a-08336161c76b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:35.824967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be425e74-f028-4fac-be5a-08336161c76b service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:35.844497 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 661/1323] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:37.877726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beeba68-74fa-4b2a-bb76-189bddd6cf04 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:37.878699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beeba68-74fa-4b2a-bb76-189bddd6cf04 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:37.878699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beeba68-74fa-4b2a-bb76-189bddd6cf04 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:37.879703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 663/1324] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:37.885780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb43a9-afdd-47ce-a824-3318783d4f42 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:37.886313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb43a9-afdd-47ce-a824-3318783d4f42 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:37.886313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb43a9-afdd-47ce-a824-3318783d4f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:37.979250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-97cb43a9-afdd-47ce-a824-3318783d4f42 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:37.980542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-97cb43a9-afdd-47ce-a824-3318783d4f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:38.095471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 662/1325] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:38.405828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b94f76-a89a-4dca-b363-5578c5cd4e46 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:38.406243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b94f76-a89a-4dca-b363-5578c5cd4e46 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:38.406532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b94f76-a89a-4dca-b363-5578c5cd4e46 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:38.408437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 664/1326] 10.222.0.27 () {54 vars in 965 bytes} [Wed Aug 31 12:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:38.417258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc49826-1f19-45f4-8ae1-0d04dda3e695 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:38.417559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc49826-1f19-45f4-8ae1-0d04dda3e695 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:38.417848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc49826-1f19-45f4-8ae1-0d04dda3e695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:38.453138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:38.511538 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ddc49826-1f19-45f4-8ae1-0d04dda3e695 None None] MFA Rules not processed for user `ba8c7348ad2e457e80e1fdd2d93aacc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:38.512768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ddc49826-1f19-45f4-8ae1-0d04dda3e695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:38.609562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 663/1327] 10.222.0.27 () {58 vars in 1062 bytes} [Wed Aug 31 12:10:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:38.619007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b94f76-a89a-4dca-b363-5578c5cd4e46 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:38.620678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d4b94f76-a89a-4dca-b363-5578c5cd4e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:38.755120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b94f76-a89a-4dca-b363-5578c5cd4e46 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:38.839543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2540986-0fde-4e50-8efa-2d0c77fe8bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:38.840065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2540986-0fde-4e50-8efa-2d0c77fe8bbd service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:38.840338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2540986-0fde-4e50-8efa-2d0c77fe8bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:38.840718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2540986-0fde-4e50-8efa-2d0c77fe8bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:38.842369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a2540986-0fde-4e50-8efa-2d0c77fe8bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:38.865581 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:38.980768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2540986-0fde-4e50-8efa-2d0c77fe8bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:38.983183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2540986-0fde-4e50-8efa-2d0c77fe8bbd service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:39.005598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 665/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:41.288643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bbf3f-867e-4d3e-a053-dfa2615e11ee None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:41.289454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bbf3f-867e-4d3e-a053-dfa2615e11ee None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:41.289917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bbf3f-867e-4d3e-a053-dfa2615e11ee None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:41.292605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 664/1329] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:41.299243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cbec7-4499-4fbe-b566-2ed72f858187 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:41.299243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cbec7-4499-4fbe-b566-2ed72f858187 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:41.299243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cbec7-4499-4fbe-b566-2ed72f858187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:41.378448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-292cbec7-4499-4fbe-b566-2ed72f858187 None None] MFA Rules not processed for user `16aab776604b46f6b9f706d4d684a6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:41.379961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-292cbec7-4499-4fbe-b566-2ed72f858187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:41.448947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:41.483262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 666/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:41.768125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bbf3f-867e-4d3e-a053-dfa2615e11ee None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:41.769901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0e0bbf3f-867e-4d3e-a053-dfa2615e11ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:41.892279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bbf3f-867e-4d3e-a053-dfa2615e11ee None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:41.917095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:41.985967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ee4fb-2573-40ea-97fa-ecd8d0428f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:41.986603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee4fb-2573-40ea-97fa-ecd8d0428f96 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:41.986879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee4fb-2573-40ea-97fa-ecd8d0428f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:41.987272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee4fb-2573-40ea-97fa-ecd8d0428f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:41.989380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5e7ee4fb-2573-40ea-97fa-ecd8d0428f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:42.125744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ee4fb-2573-40ea-97fa-ecd8d0428f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:42.128406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ee4fb-2573-40ea-97fa-ecd8d0428f96 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:42.164326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 665/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:44.591834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295009a6-9e69-4f45-85fc-c66008f04fb1 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:44.592446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295009a6-9e69-4f45-85fc-c66008f04fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:44.592446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295009a6-9e69-4f45-85fc-c66008f04fb1 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:44.593756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 667/1332] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:44.600437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9009e2-983e-47d0-abfa-cd8b9f6f4f72 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:44.601016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9009e2-983e-47d0-abfa-cd8b9f6f4f72 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:44.601016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9009e2-983e-47d0-abfa-cd8b9f6f4f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:44.679898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:44.683696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2e9009e2-983e-47d0-abfa-cd8b9f6f4f72 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:44.685341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2e9009e2-983e-47d0-abfa-cd8b9f6f4f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:44.790219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 666/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:45.069867 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295009a6-9e69-4f45-85fc-c66008f04fb1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:45.071547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-295009a6-9e69-4f45-85fc-c66008f04fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:45.180547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295009a6-9e69-4f45-85fc-c66008f04fb1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:45.260828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b728f-1875-4b87-80ff-b66ab22ca59c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:45.261458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b728f-1875-4b87-80ff-b66ab22ca59c service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:45.261734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b728f-1875-4b87-80ff-b66ab22ca59c service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:45.262004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b728f-1875-4b87-80ff-b66ab22ca59c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:45.263811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-757b728f-1875-4b87-80ff-b66ab22ca59c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:45.293358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:45.392567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b728f-1875-4b87-80ff-b66ab22ca59c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:45.394375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b728f-1875-4b87-80ff-b66ab22ca59c service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:45.414016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 668/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:47.383329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea15c-511d-4054-939d-9d3b3d0753e7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:47.383968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea15c-511d-4054-939d-9d3b3d0753e7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:47.383968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea15c-511d-4054-939d-9d3b3d0753e7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:47.385228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 667/1335] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:47.390571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf09e1-424b-4015-9aa6-9af97569ad11 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:47.391067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf09e1-424b-4015-9aa6-9af97569ad11 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:47.391067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf09e1-424b-4015-9aa6-9af97569ad11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:47.461474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4cbf09e1-424b-4015-9aa6-9af97569ad11 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:47.462702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4cbf09e1-424b-4015-9aa6-9af97569ad11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:47.546182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:47.572759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 669/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:47.857585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea15c-511d-4054-939d-9d3b3d0753e7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:47.859147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0a1ea15c-511d-4054-939d-9d3b3d0753e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:47.972648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea15c-511d-4054-939d-9d3b3d0753e7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:48.004235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:48.051898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298d50b-a4e2-4dea-8d40-f69363aa53be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:48.052500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298d50b-a4e2-4dea-8d40-f69363aa53be service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:48.052769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298d50b-a4e2-4dea-8d40-f69363aa53be service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:48.053032 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298d50b-a4e2-4dea-8d40-f69363aa53be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:48.054620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a298d50b-a4e2-4dea-8d40-f69363aa53be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:48.166450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298d50b-a4e2-4dea-8d40-f69363aa53be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:48.167831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298d50b-a4e2-4dea-8d40-f69363aa53be service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:48.187396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 668/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:50.574794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74c05-e11a-4203-85ff-fa34330f03a4 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:50.575429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74c05-e11a-4203-85ff-fa34330f03a4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:50.575429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74c05-e11a-4203-85ff-fa34330f03a4 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:50.576692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 670/1338] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:50.582377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7418e-6e63-4fbb-b53f-19a7b80137e4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:50.582818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7418e-6e63-4fbb-b53f-19a7b80137e4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:50.582931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7418e-6e63-4fbb-b53f-19a7b80137e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:50.653378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3ec7418e-6e63-4fbb-b53f-19a7b80137e4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:50.654595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3ec7418e-6e63-4fbb-b53f-19a7b80137e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:50.663118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:50.760101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 669/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:51.042080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74c05-e11a-4203-85ff-fa34330f03a4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:51.043686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-61e74c05-e11a-4203-85ff-fa34330f03a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:51.154290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74c05-e11a-4203-85ff-fa34330f03a4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:51.230750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407138dc-0222-4d99-acf9-79b27ac42769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:51.231378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407138dc-0222-4d99-acf9-79b27ac42769 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:51.231658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407138dc-0222-4d99-acf9-79b27ac42769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:51.231926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407138dc-0222-4d99-acf9-79b27ac42769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:51.233527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-407138dc-0222-4d99-acf9-79b27ac42769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:51.272680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:51.351726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407138dc-0222-4d99-acf9-79b27ac42769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:51.353167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407138dc-0222-4d99-acf9-79b27ac42769 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:51.372424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 671/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:55.291570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0958-15b3-4490-8c60-d88b43148a9c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:55.292169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0958-15b3-4490-8c60-d88b43148a9c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:55.292169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0958-15b3-4490-8c60-d88b43148a9c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:55.293466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 670/1341] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:55.298838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2fceb-7b9a-4f72-82ee-ccafa7be4898 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:55.299438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2fceb-7b9a-4f72-82ee-ccafa7be4898 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:55.299873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2fceb-7b9a-4f72-82ee-ccafa7be4898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:55.373187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6dd2fceb-7b9a-4f72-82ee-ccafa7be4898 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:55.374412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6dd2fceb-7b9a-4f72-82ee-ccafa7be4898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:55.470927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:55.485508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 672/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:55.786720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0958-15b3-4490-8c60-d88b43148a9c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:55.788299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d5ad0958-15b3-4490-8c60-d88b43148a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:55.899960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0958-15b3-4490-8c60-d88b43148a9c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:55.940510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:55.980312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf36088-c5d0-49a2-9646-56ddeaeda7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:55.980896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf36088-c5d0-49a2-9646-56ddeaeda7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:55.981226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf36088-c5d0-49a2-9646-56ddeaeda7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:55.981492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf36088-c5d0-49a2-9646-56ddeaeda7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:55.983091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dbf36088-c5d0-49a2-9646-56ddeaeda7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:56.100588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf36088-c5d0-49a2-9646-56ddeaeda7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:56.101954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf36088-c5d0-49a2-9646-56ddeaeda7b4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:56.120301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 671/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:10:58.615281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babc8f5-bb15-4257-84f0-5b839b10649f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:58.615893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babc8f5-bb15-4257-84f0-5b839b10649f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:58.615893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babc8f5-bb15-4257-84f0-5b839b10649f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:58.617233 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 673/1344] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:10:58.622998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525419c-5590-442b-a9f1-762843eb7362 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:58.623665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525419c-5590-442b-a9f1-762843eb7362 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:58.623665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525419c-5590-442b-a9f1-762843eb7362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:58.702019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3525419c-5590-442b-a9f1-762843eb7362 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:10:58.703404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3525419c-5590-442b-a9f1-762843eb7362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:58.724709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:58.810467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 672/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:10:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:10:59.089976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babc8f5-bb15-4257-84f0-5b839b10649f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:10:59.091716 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7babc8f5-bb15-4257-84f0-5b839b10649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:59.203904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babc8f5-bb15-4257-84f0-5b839b10649f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:10:59.281394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bb14f-7e1e-44df-823f-12f8d35aac63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:10:59.282002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb14f-7e1e-44df-823f-12f8d35aac63 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:10:59.282286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb14f-7e1e-44df-823f-12f8d35aac63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:10:59.282518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb14f-7e1e-44df-823f-12f8d35aac63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:10:59.284394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-673bb14f-7e1e-44df-823f-12f8d35aac63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:10:59.336818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:10:59.413655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bb14f-7e1e-44df-823f-12f8d35aac63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:10:59.415101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bb14f-7e1e-44df-823f-12f8d35aac63 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:10:59.434353 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 674/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:10:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:03.454204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c60d-64d6-4d40-9e76-ebd0f13bb328 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:03.454781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c60d-64d6-4d40-9e76-ebd0f13bb328 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:03.454781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c60d-64d6-4d40-9e76-ebd0f13bb328 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:03.456753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 673/1347] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:03.461927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e261d9-fe42-492a-bd6e-bb299e63f87a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:03.462430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e261d9-fe42-492a-bd6e-bb299e63f87a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:03.462556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e261d9-fe42-492a-bd6e-bb299e63f87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:03.547438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-80e261d9-fe42-492a-bd6e-bb299e63f87a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:03.549785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-80e261d9-fe42-492a-bd6e-bb299e63f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:03.651833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:03.655457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 675/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:03.939563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c60d-64d6-4d40-9e76-ebd0f13bb328 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:03.941993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-84a1c60d-64d6-4d40-9e76-ebd0f13bb328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:04.054414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c60d-64d6-4d40-9e76-ebd0f13bb328 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:04.105056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:04.133017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe60ee0-9953-4152-b7c2-1c80b991425c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:04.133622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60ee0-9953-4152-b7c2-1c80b991425c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:04.133890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60ee0-9953-4152-b7c2-1c80b991425c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:04.134154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60ee0-9953-4152-b7c2-1c80b991425c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:04.135748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ebe60ee0-9953-4152-b7c2-1c80b991425c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:04.259115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe60ee0-9953-4152-b7c2-1c80b991425c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:04.260599 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe60ee0-9953-4152-b7c2-1c80b991425c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:04.279956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 674/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:09.198395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e48c-2b24-4030-862f-fe63f72b7b5a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:09.198946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e48c-2b24-4030-862f-fe63f72b7b5a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:09.198946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e48c-2b24-4030-862f-fe63f72b7b5a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:09.202480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 676/1350] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:11:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:09.207974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2848b-a65e-4b3e-97e2-11977895c653 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:09.208476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2848b-a65e-4b3e-97e2-11977895c653 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:09.208476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2848b-a65e-4b3e-97e2-11977895c653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:09.315649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-88a2848b-a65e-4b3e-97e2-11977895c653 None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:09.317546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-88a2848b-a65e-4b3e-97e2-11977895c653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:09.357243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:09.482598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 675/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:11:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:13.643023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda263-1d67-4ad8-a235-e97a9d5c90c8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:13.643770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda263-1d67-4ad8-a235-e97a9d5c90c8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:13.643770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eda263-1d67-4ad8-a235-e97a9d5c90c8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:13.645030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 677/1352] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:13.650225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbda29-10af-4a1b-860c-e09307c53e51 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:13.651088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbda29-10af-4a1b-860c-e09307c53e51 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:13.651592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbda29-10af-4a1b-860c-e09307c53e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:13.743940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9edbda29-10af-4a1b-860c-e09307c53e51 None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:13.745751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9edbda29-10af-4a1b-860c-e09307c53e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:13.829000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:13.873877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 676/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:11:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:15.368244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6c78-eee0-4ecf-b337-f03c470d02bc None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:15.368810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6c78-eee0-4ecf-b337-f03c470d02bc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:15.368960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6c78-eee0-4ecf-b337-f03c470d02bc None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:15.370609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 678/1354] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:15.377166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a500-89bd-424a-813c-15b34ae9bef8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:15.377934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a500-89bd-424a-813c-15b34ae9bef8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:15.377934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a500-89bd-424a-813c-15b34ae9bef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:15.458271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7c56a500-89bd-424a-813c-15b34ae9bef8 None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:15.459522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7c56a500-89bd-424a-813c-15b34ae9bef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:15.569726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 677/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:11:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:19.131268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc13ce-ba3b-4040-afea-d67269812042 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:19.131927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc13ce-ba3b-4040-afea-d67269812042 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:19.131927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc13ce-ba3b-4040-afea-d67269812042 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:19.133199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 679/1356] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:19.141301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907749-c545-40cf-a2d8-436478416eb2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:19.141847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907749-c545-40cf-a2d8-436478416eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:19.141965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907749-c545-40cf-a2d8-436478416eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:19.191786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:19.233228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9e907749-c545-40cf-a2d8-436478416eb2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:19.234771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9e907749-c545-40cf-a2d8-436478416eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:19.342093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 678/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:19.721575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64280d9-98d4-4e95-b923-dfbe055af6f6 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:19.722230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64280d9-98d4-4e95-b923-dfbe055af6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:19.722769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64280d9-98d4-4e95-b923-dfbe055af6f6 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:19.724931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 680/1358] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 31 12:11:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:19.732620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50374838-0bea-45a4-b342-c908f30f65b0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:19.733055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50374838-0bea-45a4-b342-c908f30f65b0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:19.733581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50374838-0bea-45a4-b342-c908f30f65b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:19.835756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-50374838-0bea-45a4-b342-c908f30f65b0 None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:19.837309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-50374838-0bea-45a4-b342-c908f30f65b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:19.909240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:20.004834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 679/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 31 12:11:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:20.015585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64280d9-98d4-4e95-b923-dfbe055af6f6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:20.017481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e64280d9-98d4-4e95-b923-dfbe055af6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:20.201578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64280d9-98d4-4e95-b923-dfbe055af6f6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:20.297948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:20.352995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad94c1-29c0-4da2-97a9-28697f4274aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:20.353872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad94c1-29c0-4da2-97a9-28697f4274aa service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:20.354286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad94c1-29c0-4da2-97a9-28697f4274aa service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:20.354840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad94c1-29c0-4da2-97a9-28697f4274aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:20.356939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fcad94c1-29c0-4da2-97a9-28697f4274aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:20.476511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad94c1-29c0-4da2-97a9-28697f4274aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:20.478385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad94c1-29c0-4da2-97a9-28697f4274aa service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:20.503366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 681/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:22.690581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5db438-2dca-401e-8544-4f5fb520b781 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:22.691033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5db438-2dca-401e-8544-4f5fb520b781 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:22.691368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5db438-2dca-401e-8544-4f5fb520b781 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:22.695280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 680/1361] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:22.698459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317e19-f1db-48b0-b16a-09e74d543f3c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:22.698860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317e19-f1db-48b0-b16a-09e74d543f3c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:22.699186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317e19-f1db-48b0-b16a-09e74d543f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:22.771493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7e317e19-f1db-48b0-b16a-09e74d543f3c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:22.772618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7e317e19-f1db-48b0-b16a-09e74d543f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:22.812587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:22.885858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 682/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:23.173512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5db438-2dca-401e-8544-4f5fb520b781 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:23.175450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3a5db438-2dca-401e-8544-4f5fb520b781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:23.357805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5db438-2dca-401e-8544-4f5fb520b781 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:23.369627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:23.502577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb32316-a8ff-484b-bfcf-1e5644f926ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:23.503244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb32316-a8ff-484b-bfcf-1e5644f926ca service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:23.503732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb32316-a8ff-484b-bfcf-1e5644f926ca service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:23.504067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb32316-a8ff-484b-bfcf-1e5644f926ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:23.505736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4bb32316-a8ff-484b-bfcf-1e5644f926ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:23.604118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb32316-a8ff-484b-bfcf-1e5644f926ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:23.605752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb32316-a8ff-484b-bfcf-1e5644f926ca service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:23.628753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 681/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:25.849711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94d3ce-4aaf-4291-9c65-055e2bde759d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:25.850492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94d3ce-4aaf-4291-9c65-055e2bde759d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:25.850492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94d3ce-4aaf-4291-9c65-055e2bde759d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:25.851784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 683/1364] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:11:25.858370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878d7d1-5586-4630-af60-82bb0ee185b4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:25.858902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878d7d1-5586-4630-af60-82bb0ee185b4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:25.859027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878d7d1-5586-4630-af60-82bb0ee185b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:25.886366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:25.972582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4878d7d1-5586-4630-af60-82bb0ee185b4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:25.974205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4878d7d1-5586-4630-af60-82bb0ee185b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:26.159699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 682/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:26.462279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e55d6-f599-42aa-8798-7d7917c17f7b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:26.462839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e55d6-f599-42aa-8798-7d7917c17f7b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:26.463107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e55d6-f599-42aa-8798-7d7917c17f7b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:26.464756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 684/1366] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 31 12:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:11:26.474735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d8af4-7ac2-46fe-8a31-a438556de395 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:26.475610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d8af4-7ac2-46fe-8a31-a438556de395 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:26.476154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d8af4-7ac2-46fe-8a31-a438556de395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:26.629513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-366d8af4-7ac2-46fe-8a31-a438556de395 None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:26.631631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-366d8af4-7ac2-46fe-8a31-a438556de395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:26.702892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:26.859286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 683/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 31 12:11:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:26.879043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e55d6-f599-42aa-8798-7d7917c17f7b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:26.886629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-db8e55d6-f599-42aa-8798-7d7917c17f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:27.199436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e55d6-f599-42aa-8798-7d7917c17f7b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:27.241844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:27.418247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61eb00-0f56-4363-90c6-877dec49729c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:27.418750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61eb00-0f56-4363-90c6-877dec49729c service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:27.419166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61eb00-0f56-4363-90c6-877dec49729c service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:27.419478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61eb00-0f56-4363-90c6-877dec49729c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:27.421172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dd61eb00-0f56-4363-90c6-877dec49729c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:27.582425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61eb00-0f56-4363-90c6-877dec49729c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:27.585596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61eb00-0f56-4363-90c6-877dec49729c service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:27.624008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 685/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:28.605401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a59ba-6752-4178-83c1-b73b5fc63003 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:28.606218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a59ba-6752-4178-83c1-b73b5fc63003 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:28.606218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a59ba-6752-4178-83c1-b73b5fc63003 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:28.607742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 684/1369] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:11:28.613861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dc90-6a3f-4321-b66e-f277122561b6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:28.614675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dc90-6a3f-4321-b66e-f277122561b6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:28.614675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dc90-6a3f-4321-b66e-f277122561b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:28.653930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:28.705169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-61f7dc90-6a3f-4321-b66e-f277122561b6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:28.706361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-61f7dc90-6a3f-4321-b66e-f277122561b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:28.873986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 686/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:11:29.538362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a82b7-9e1c-4220-bf56-e0b2a0b6d394 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:29.538692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a82b7-9e1c-4220-bf56-e0b2a0b6d394 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:29.538916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a82b7-9e1c-4220-bf56-e0b2a0b6d394 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:29.540355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 685/1371] 10.222.0.27 () {54 vars in 964 bytes} [Wed Aug 31 12:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:29.546926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57235ba-3e0d-4cff-9c84-663de08408d8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:29.547480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57235ba-3e0d-4cff-9c84-663de08408d8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:29.548014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57235ba-3e0d-4cff-9c84-663de08408d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:29.692470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d57235ba-3e0d-4cff-9c84-663de08408d8 None None] MFA Rules not processed for user `db3393d3cb0e4bbcb21e403c431bf61b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:29.694169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d57235ba-3e0d-4cff-9c84-663de08408d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:29.785013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:29.824505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 687/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Aug 31 12:11:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:29.832741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a82b7-9e1c-4220-bf56-e0b2a0b6d394 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:29.834455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7e2a82b7-9e1c-4220-bf56-e0b2a0b6d394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:29.959627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a82b7-9e1c-4220-bf56-e0b2a0b6d394 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:30.006939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:30.050848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:30.051554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:30.051914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:30.052213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:30.053996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:30.231940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:30.233447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:30.236970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac823c-d9b0-4264-ad20-49ca985a54d8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:30.237258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac823c-d9b0-4264-ad20-49ca985a54d8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:30.237534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac823c-d9b0-4264-ad20-49ca985a54d8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:30.238935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 688/1373] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:11:30.245839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517d7a-d65e-48cb-b681-87417a3ef210 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:30.246318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517d7a-d65e-48cb-b681-87417a3ef210 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:30.246796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517d7a-d65e-48cb-b681-87417a3ef210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:30.255745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 686/1374] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:11:30.362380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5a517d7a-d65e-48cb-b681-87417a3ef210 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:30.363886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5a517d7a-d65e-48cb-b681-87417a3ef210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:30.478608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 689/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:11:31.002994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:31.005193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:31.111464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:31.157389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922bf46-b5b0-44d8-80a3-0ba91341b477 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:31.320277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593688d-2d9f-430a-ba3f-9938f9e708e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:31.321308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593688d-2d9f-430a-ba3f-9938f9e708e8 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:31.321791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593688d-2d9f-430a-ba3f-9938f9e708e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:31.322272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593688d-2d9f-430a-ba3f-9938f9e708e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:31.325398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8593688d-2d9f-430a-ba3f-9938f9e708e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:31.483400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:31.502896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593688d-2d9f-430a-ba3f-9938f9e708e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:31.505540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593688d-2d9f-430a-ba3f-9938f9e708e8 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:31.530793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 687/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:33.723961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba63dc-2e9f-48ab-b4dd-7f661b25061c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:33.724733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba63dc-2e9f-48ab-b4dd-7f661b25061c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:33.724733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba63dc-2e9f-48ab-b4dd-7f661b25061c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:33.726639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 690/1377] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:33.734154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5bd3d-f3f4-4e53-bcf3-7cd281bf0fb7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:33.734785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5bd3d-f3f4-4e53-bcf3-7cd281bf0fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:33.734785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5bd3d-f3f4-4e53-bcf3-7cd281bf0fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:33.821613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-52e5bd3d-f3f4-4e53-bcf3-7cd281bf0fb7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:33.822836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-52e5bd3d-f3f4-4e53-bcf3-7cd281bf0fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:34.019392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 688/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:34.408378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba63dc-2e9f-48ab-b4dd-7f661b25061c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:34.410251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-71ba63dc-2e9f-48ab-b4dd-7f661b25061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:34.497175 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:34.645493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba63dc-2e9f-48ab-b4dd-7f661b25061c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:34.774433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae5e96-535b-4d7b-a120-cc497aa7e407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:34.775536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5e96-535b-4d7b-a120-cc497aa7e407 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:34.775536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5e96-535b-4d7b-a120-cc497aa7e407 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:34.775738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5e96-535b-4d7b-a120-cc497aa7e407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:34.777255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3aae5e96-535b-4d7b-a120-cc497aa7e407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:34.791485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:34.916966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae5e96-535b-4d7b-a120-cc497aa7e407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:34.919759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae5e96-535b-4d7b-a120-cc497aa7e407 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:34.955800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 691/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:37.586870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c1927-4111-4581-a1ff-2ed9bedf5c88 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:37.587735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c1927-4111-4581-a1ff-2ed9bedf5c88 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:37.587735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c1927-4111-4581-a1ff-2ed9bedf5c88 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:37.589094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 689/1380] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:11:37.595479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaeb50-0aa6-4a13-9a6e-09b25e67e5f8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:37.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaeb50-0aa6-4a13-9a6e-09b25e67e5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:37.596074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caaeb50-0aa6-4a13-9a6e-09b25e67e5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:37.671268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7caaeb50-0aa6-4a13-9a6e-09b25e67e5f8 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:37.672539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7caaeb50-0aa6-4a13-9a6e-09b25e67e5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:37.794890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:37.812405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 692/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:11:38.118930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c1927-4111-4581-a1ff-2ed9bedf5c88 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:38.121067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2c4c1927-4111-4581-a1ff-2ed9bedf5c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:38.216996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:38.392121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c1927-4111-4581-a1ff-2ed9bedf5c88 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:38.611730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270de90-1694-4467-b5a0-38cf816ebf1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:38.612693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270de90-1694-4467-b5a0-38cf816ebf1a service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:38.612693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270de90-1694-4467-b5a0-38cf816ebf1a service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:38.613196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270de90-1694-4467-b5a0-38cf816ebf1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:38.615290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a270de90-1694-4467-b5a0-38cf816ebf1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:38.655344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:38.762163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270de90-1694-4467-b5a0-38cf816ebf1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:38.764015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270de90-1694-4467-b5a0-38cf816ebf1a service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:38.792416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 690/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:41.284478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dce43-9f96-4a3e-aef8-afa0d1b9142e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:41.285309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dce43-9f96-4a3e-aef8-afa0d1b9142e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:41.285489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dce43-9f96-4a3e-aef8-afa0d1b9142e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:41.289144 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 693/1383] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:41.297313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c882d1-6ba0-4f58-a45c-14986fc37ea3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:41.297951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c882d1-6ba0-4f58-a45c-14986fc37ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:41.298240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c882d1-6ba0-4f58-a45c-14986fc37ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:41.420821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-29c882d1-6ba0-4f58-a45c-14986fc37ea3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:41.422715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-29c882d1-6ba0-4f58-a45c-14986fc37ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:41.551100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:41.560025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 691/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:41.877270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dce43-9f96-4a3e-aef8-afa0d1b9142e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:41.879632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-293dce43-9f96-4a3e-aef8-afa0d1b9142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:42.094655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dce43-9f96-4a3e-aef8-afa0d1b9142e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:42.182640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:42.267943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5717ee-471c-4be5-a622-76710b5ece07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:42.268655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5717ee-471c-4be5-a622-76710b5ece07 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:42.269031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5717ee-471c-4be5-a622-76710b5ece07 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:42.269395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5717ee-471c-4be5-a622-76710b5ece07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:42.271392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6c5717ee-471c-4be5-a622-76710b5ece07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:42.402166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5717ee-471c-4be5-a622-76710b5ece07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:42.407110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5717ee-471c-4be5-a622-76710b5ece07 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:42.449674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 694/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:44.540282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb08e2-d9b9-4fe4-8232-033a696e2f4b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:44.541752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb08e2-d9b9-4fe4-8232-033a696e2f4b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:44.542055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb08e2-d9b9-4fe4-8232-033a696e2f4b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:44.544672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 692/1386] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:44.550876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d9313-66aa-4997-913a-a0e12fdbdb29 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:44.551720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d9313-66aa-4997-913a-a0e12fdbdb29 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:44.551720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d9313-66aa-4997-913a-a0e12fdbdb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:44.669301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-112d9313-66aa-4997-913a-a0e12fdbdb29 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:44.671664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-112d9313-66aa-4997-913a-a0e12fdbdb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:44.710244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:44.830539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 695/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:45.141328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb08e2-d9b9-4fe4-8232-033a696e2f4b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:45.142943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c6fb08e2-d9b9-4fe4-8232-033a696e2f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:45.304650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb08e2-d9b9-4fe4-8232-033a696e2f4b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:45.390321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:45.445970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0dacc-ad07-4fa1-bc7d-91e235ddaec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:45.446745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0dacc-ad07-4fa1-bc7d-91e235ddaec7 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:45.447019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0dacc-ad07-4fa1-bc7d-91e235ddaec7 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:45.447330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0dacc-ad07-4fa1-bc7d-91e235ddaec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:45.449165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4db0dacc-ad07-4fa1-bc7d-91e235ddaec7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:45.562330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0dacc-ad07-4fa1-bc7d-91e235ddaec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:45.564034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0dacc-ad07-4fa1-bc7d-91e235ddaec7 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:45.585079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 693/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:46.892570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0f806-d23e-41e4-87c0-e3464471f613 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:46.893232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0f806-d23e-41e4-87c0-e3464471f613 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:46.893232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0f806-d23e-41e4-87c0-e3464471f613 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:46.894751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 696/1389] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:46.900625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2250-9749-4474-95a9-2fe6654fa83d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:46.901272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2250-9749-4474-95a9-2fe6654fa83d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:46.901384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2250-9749-4474-95a9-2fe6654fa83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:46.989677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6b3c2250-9749-4474-95a9-2fe6654fa83d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:46.990872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6b3c2250-9749-4474-95a9-2fe6654fa83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:47.030062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:47.095831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 694/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:11:47.403491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c079d-607e-47b1-8e98-8464bd717837 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:47.403707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c079d-607e-47b1-8e98-8464bd717837 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:47.403948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c079d-607e-47b1-8e98-8464bd717837 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:47.405465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 697/1391] 10.222.0.27 () {54 vars in 967 bytes} [Wed Aug 31 12:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:11:47.411166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7af5b-ea63-4796-ae44-5921c6b5697c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:47.411692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7af5b-ea63-4796-ae44-5921c6b5697c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:47.412037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7af5b-ea63-4796-ae44-5921c6b5697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:47.515478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ccb7af5b-ea63-4796-ae44-5921c6b5697c None None] MFA Rules not processed for user `eef756349e5844e59d7b20072f63c41c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:47.516722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ccb7af5b-ea63-4796-ae44-5921c6b5697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:47.620114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 695/1392] 10.222.0.27 () {58 vars in 1064 bytes} [Wed Aug 31 12:11:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:47.628255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c079d-607e-47b1-8e98-8464bd717837 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:47.629932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c47c079d-607e-47b1-8e98-8464bd717837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:47.647634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41ba2-447a-4494-9741-5376a5365ac9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:47.648305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41ba2-447a-4494-9741-5376a5365ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:47.648305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41ba2-447a-4494-9741-5376a5365ac9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:47.649612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 696/1393] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:11:47.655178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee71796-d380-46da-8d35-1e003c9b1ca2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:47.655476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee71796-d380-46da-8d35-1e003c9b1ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:47.655713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee71796-d380-46da-8d35-1e003c9b1ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:47.697571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:47.738651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cee71796-d380-46da-8d35-1e003c9b1ca2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:47.739925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cee71796-d380-46da-8d35-1e003c9b1ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:47.754964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c079d-607e-47b1-8e98-8464bd717837 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:47.831335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:47.857082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 697/1394] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:11:47.859957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35890e4f-ff4f-4697-9b45-06536a2eff4b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:47.859957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890e4f-ff4f-4697-9b45-06536a2eff4b service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:47.860726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890e4f-ff4f-4697-9b45-06536a2eff4b service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:47.860726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890e4f-ff4f-4697-9b45-06536a2eff4b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:47.862215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-35890e4f-ff4f-4697-9b45-06536a2eff4b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:48.017670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35890e4f-ff4f-4697-9b45-06536a2eff4b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:48.020571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35890e4f-ff4f-4697-9b45-06536a2eff4b service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:48.045216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 698/1395] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:48.163540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee71796-d380-46da-8d35-1e003c9b1ca2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:48.165183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cee71796-d380-46da-8d35-1e003c9b1ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:48.328982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:48.333999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee71796-d380-46da-8d35-1e003c9b1ca2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:48.522583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48862f0c-d6fe-4aed-a569-dde411528d35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:48.523614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862f0c-d6fe-4aed-a569-dde411528d35 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:48.524063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862f0c-d6fe-4aed-a569-dde411528d35 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:48.524515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862f0c-d6fe-4aed-a569-dde411528d35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:48.527550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-48862f0c-d6fe-4aed-a569-dde411528d35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:48.656849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48862f0c-d6fe-4aed-a569-dde411528d35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:48.658533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48862f0c-d6fe-4aed-a569-dde411528d35 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:48.681746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 698/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:48.872806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cc899-03d2-472a-ae8e-e87879ab210c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:48.873580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cc899-03d2-472a-ae8e-e87879ab210c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:48.874158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cc899-03d2-472a-ae8e-e87879ab210c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:48.974853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-886cc899-03d2-472a-ae8e-e87879ab210c None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:48.976297 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-886cc899-03d2-472a-ae8e-e87879ab210c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:49.019553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:49.087570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 699/1397] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:11:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:49.142583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fb70a-a4ee-470a-8565-9d45277a1423 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:49.143472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fb70a-a4ee-470a-8565-9d45277a1423 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:49.143851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fb70a-a4ee-470a-8565-9d45277a1423 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:49.145926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 699/1398] 10.222.0.27 () {54 vars in 943 bytes} [Wed Aug 31 12:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:49.152663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4dd00-4a62-4026-83f7-733cd69b0bb4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:49.153113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4dd00-4a62-4026-83f7-733cd69b0bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:49.153614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4dd00-4a62-4026-83f7-733cd69b0bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:49.272371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-98c4dd00-4a62-4026-83f7-733cd69b0bb4 None None] MFA Rules not processed for user `981e9da7f2774c90a667fab35ecab64b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:49.273599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-98c4dd00-4a62-4026-83f7-733cd69b0bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:49.396677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 700/1399] 10.222.0.27 () {58 vars in 1040 bytes} [Wed Aug 31 12:11:49 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:49.411795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fb70a-a4ee-470a-8565-9d45277a1423 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:49.414597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0a3fb70a-a4ee-470a-8565-9d45277a1423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:49.427068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:49.622841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fb70a-a4ee-470a-8565-9d45277a1423 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:49.726009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:49.753711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285b01a-eb3a-47e6-b9d5-54235236d907 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:49.754420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b01a-eb3a-47e6-b9d5-54235236d907 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:49.754721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b01a-eb3a-47e6-b9d5-54235236d907 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:49.755105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b01a-eb3a-47e6-b9d5-54235236d907 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:49.757547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4285b01a-eb3a-47e6-b9d5-54235236d907 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:49.880063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285b01a-eb3a-47e6-b9d5-54235236d907 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:49.881880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285b01a-eb3a-47e6-b9d5-54235236d907 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:49.894892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 700/1400] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:11:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:11:50.790632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec444a5-5d61-4cc7-b572-bf02f95c3d7e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:50.791466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec444a5-5d61-4cc7-b572-bf02f95c3d7e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:50.791466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec444a5-5d61-4cc7-b572-bf02f95c3d7e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:50.792701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 701/1401] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:50.799276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d4185-205f-4399-8920-9b17b5aad532 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:50.800208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d4185-205f-4399-8920-9b17b5aad532 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:50.800208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d4185-205f-4399-8920-9b17b5aad532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:50.910201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c89d4185-205f-4399-8920-9b17b5aad532 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:50.912103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c89d4185-205f-4399-8920-9b17b5aad532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:51.004197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:51.034049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 701/1402] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:51.351716 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec444a5-5d61-4cc7-b572-bf02f95c3d7e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:51.353705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1ec444a5-5d61-4cc7-b572-bf02f95c3d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:51.363131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:51.560335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec444a5-5d61-4cc7-b572-bf02f95c3d7e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:51.661972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:51.712872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c06ea-2b9b-43b5-8d1a-97757dc67ea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:51.713688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c06ea-2b9b-43b5-8d1a-97757dc67ea8 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:51.713973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c06ea-2b9b-43b5-8d1a-97757dc67ea8 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:51.714502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c06ea-2b9b-43b5-8d1a-97757dc67ea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:51.716707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9f8c06ea-2b9b-43b5-8d1a-97757dc67ea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:51.842306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c06ea-2b9b-43b5-8d1a-97757dc67ea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:51.844040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c06ea-2b9b-43b5-8d1a-97757dc67ea8 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:51.867336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 702/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:54.058095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf9f72-9875-4061-8e46-0531de56b9b1 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:54.058802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf9f72-9875-4061-8e46-0531de56b9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:54.059622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf9f72-9875-4061-8e46-0531de56b9b1 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:54.062655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 702/1404] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:54.068331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af912d71-aa99-44fb-a28c-b041cb6fc4e6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:54.069159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af912d71-aa99-44fb-a28c-b041cb6fc4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:54.069159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af912d71-aa99-44fb-a28c-b041cb6fc4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:54.147525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-af912d71-aa99-44fb-a28c-b041cb6fc4e6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:54.148770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-af912d71-aa99-44fb-a28c-b041cb6fc4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:54.196873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:54.261931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 703/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:54.557183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf9f72-9875-4061-8e46-0531de56b9b1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:54.559049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9dcf9f72-9875-4061-8e46-0531de56b9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:54.791817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf9f72-9875-4061-8e46-0531de56b9b1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:54.898422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:54.988152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c4d42-1f80-4512-98e7-6edb1bb76fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:54.989560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c4d42-1f80-4512-98e7-6edb1bb76fa1 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:54.990093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c4d42-1f80-4512-98e7-6edb1bb76fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:54.990364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c4d42-1f80-4512-98e7-6edb1bb76fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:54.992420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-891c4d42-1f80-4512-98e7-6edb1bb76fa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:55.154373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c4d42-1f80-4512-98e7-6edb1bb76fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:55.157510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c4d42-1f80-4512-98e7-6edb1bb76fa1 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:55.193642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 703/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:11:57.384565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1570f2f-c3aa-4983-bd4a-2bf86bbe8116 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:57.385530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1570f2f-c3aa-4983-bd4a-2bf86bbe8116 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:57.385810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1570f2f-c3aa-4983-bd4a-2bf86bbe8116 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:57.387937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 704/1407] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:11:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:11:57.393610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beffa1e-072d-4a75-b2c8-c5948c607138 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:57.394170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beffa1e-072d-4a75-b2c8-c5948c607138 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:57.394170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beffa1e-072d-4a75-b2c8-c5948c607138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:57.472783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9beffa1e-072d-4a75-b2c8-c5948c607138 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:11:57.476051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9beffa1e-072d-4a75-b2c8-c5948c607138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:57.496109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:57.595875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 704/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:11:57.891270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1570f2f-c3aa-4983-bd4a-2bf86bbe8116 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:11:57.891270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c1570f2f-c3aa-4983-bd4a-2bf86bbe8116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:58.076038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1570f2f-c3aa-4983-bd4a-2bf86bbe8116 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:11:58.109035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:11:58.215859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd4a2-d30b-4d51-8891-f18167f894c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:11:58.216686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd4a2-d30b-4d51-8891-f18167f894c1 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:11:58.216850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd4a2-d30b-4d51-8891-f18167f894c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:11:58.217238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd4a2-d30b-4d51-8891-f18167f894c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:11:58.218939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5d8fd4a2-d30b-4d51-8891-f18167f894c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:11:58.327611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fd4a2-d30b-4d51-8891-f18167f894c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:11:58.329356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fd4a2-d30b-4d51-8891-f18167f894c1 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:11:58.352677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 705/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:07.593138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=36 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:08.349578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:12.696706 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294976-3750-4b2f-88bb-4bde021acb96 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:12.697508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294976-3750-4b2f-88bb-4bde021acb96 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:12.697508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294976-3750-4b2f-88bb-4bde021acb96 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:12.698608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 705/1410] 10.222.0.27 () {54 vars in 918 bytes} [Wed Aug 31 12:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:12.704340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c081bf-8c46-4644-a41b-8fddf834d4e4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:12.704807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c081bf-8c46-4644-a41b-8fddf834d4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:12.704935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c081bf-8c46-4644-a41b-8fddf834d4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:12.788948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-16c081bf-8c46-4644-a41b-8fddf834d4e4 None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:12.790190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-16c081bf-8c46-4644-a41b-8fddf834d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:12.913806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 706/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Aug 31 12:12:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:12:12.934293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca43f3e-16ae-41fd-995b-90820cbd8583 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:12.937946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca43f3e-16ae-41fd-995b-90820cbd8583 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:12.937946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca43f3e-16ae-41fd-995b-90820cbd8583 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:12.937946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 706/1412] 10.222.0.27 () {54 vars in 918 bytes} [Wed Aug 31 12:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:12.941934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4568c84-e057-4980-8744-9fe49293f80d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:12.942213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4568c84-e057-4980-8744-9fe49293f80d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:12.942624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4568c84-e057-4980-8744-9fe49293f80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:13.028752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:13.033970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a4568c84-e057-4980-8744-9fe49293f80d None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:13.037797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a4568c84-e057-4980-8744-9fe49293f80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:13.170828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 707/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Aug 31 12:12:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:12:15.584520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86734b75-9226-4969-a0e5-ebf5095020e3 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:15.585126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86734b75-9226-4969-a0e5-ebf5095020e3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:15.585126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86734b75-9226-4969-a0e5-ebf5095020e3 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:15.586377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:15.594156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb2007-30c1-4f22-acb4-c78dde466331 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:15.594982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb2007-30c1-4f22-acb4-c78dde466331 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:15.594982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb2007-30c1-4f22-acb4-c78dde466331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:15.674870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-27fb2007-30c1-4f22-acb4-c78dde466331 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:15.676126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-27fb2007-30c1-4f22-acb4-c78dde466331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:15.735728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:15.784416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:16.074820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86734b75-9226-4969-a0e5-ebf5095020e3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:16.076458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-86734b75-9226-4969-a0e5-ebf5095020e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:16.246136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86734b75-9226-4969-a0e5-ebf5095020e3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:16.327934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:16.371551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a925f23-8c1b-4faf-8932-0178356b880b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:16.372243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a925f23-8c1b-4faf-8932-0178356b880b service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:16.372498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a925f23-8c1b-4faf-8932-0178356b880b service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:16.372787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a925f23-8c1b-4faf-8932-0178356b880b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:16.374419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a925f23-8c1b-4faf-8932-0178356b880b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:16.476524 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a925f23-8c1b-4faf-8932-0178356b880b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:16.479175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a925f23-8c1b-4faf-8932-0178356b880b service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:16.502146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 708/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:17.414278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7e88-171e-4501-b3dc-543378cac0d3 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:17.414855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7e88-171e-4501-b3dc-543378cac0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:17.414855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7e88-171e-4501-b3dc-543378cac0d3 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:17.416370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:12:17.422162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd27f9b-1e71-4183-bc0e-fd20dba9a748 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:17.422698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd27f9b-1e71-4183-bc0e-fd20dba9a748 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:17.422698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd27f9b-1e71-4183-bc0e-fd20dba9a748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:17.507508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2cd27f9b-1e71-4183-bc0e-fd20dba9a748 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:17.508775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2cd27f9b-1e71-4183-bc0e-fd20dba9a748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:17.537787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:17.615144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:17.765395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed7e88-171e-4501-b3dc-543378cac0d3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:17.766934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8aed7e88-171e-4501-b3dc-543378cac0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:17.929358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed7e88-171e-4501-b3dc-543378cac0d3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:17.977493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:18.067872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d1f21-327c-482b-8d2f-7725e93de1ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:18.068524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1f21-327c-482b-8d2f-7725e93de1ca service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:18.068824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1f21-327c-482b-8d2f-7725e93de1ca service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:18.069093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d1f21-327c-482b-8d2f-7725e93de1ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:18.070733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-514d1f21-327c-482b-8d2f-7725e93de1ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:18.206752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d1f21-327c-482b-8d2f-7725e93de1ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:18.208429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d1f21-327c-482b-8d2f-7725e93de1ca service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:18.232532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 710/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:19.273439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4363-3456-47de-a702-9ca09b72d8fb None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:19.274024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4363-3456-47de-a702-9ca09b72d8fb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:19.274183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4363-3456-47de-a702-9ca09b72d8fb None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:19.275700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:19.281495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270910e-f06d-4b87-879b-56caead42f0e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:19.282006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270910e-f06d-4b87-879b-56caead42f0e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:19.282006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270910e-f06d-4b87-879b-56caead42f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:19.351497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:19.386959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d270910e-f06d-4b87-879b-56caead42f0e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:19.389059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d270910e-f06d-4b87-879b-56caead42f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:19.503022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:19.656458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb4363-3456-47de-a702-9ca09b72d8fb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:19.658078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-05eb4363-3456-47de-a702-9ca09b72d8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:19.840286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb4363-3456-47de-a702-9ca09b72d8fb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:19.853059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:19.970249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3508a5-b7c3-4b82-9ea0-fdcedade19d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:19.970905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3508a5-b7c3-4b82-9ea0-fdcedade19d4 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:19.971111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3508a5-b7c3-4b82-9ea0-fdcedade19d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:19.971420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3508a5-b7c3-4b82-9ea0-fdcedade19d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:19.973144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3c3508a5-b7c3-4b82-9ea0-fdcedade19d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:20.068462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3508a5-b7c3-4b82-9ea0-fdcedade19d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:20.070682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3508a5-b7c3-4b82-9ea0-fdcedade19d4 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:20.091677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 711/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:20.977884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a158-e9f1-4464-a048-cfa9cba9c96c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:20.978588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a158-e9f1-4464-a048-cfa9cba9c96c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:20.978588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a158-e9f1-4464-a048-cfa9cba9c96c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:20.979923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:12:20.985826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf82d7-df3f-4532-86cd-072e026a7b35 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:20.986544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf82d7-df3f-4532-86cd-072e026a7b35 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:20.986544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf82d7-df3f-4532-86cd-072e026a7b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:21.009067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:21.072456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-99bf82d7-df3f-4532-86cd-072e026a7b35 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:21.073592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-99bf82d7-df3f-4532-86cd-072e026a7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:21.179785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:21.335626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16a158-e9f1-4464-a048-cfa9cba9c96c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:21.338021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5e16a158-e9f1-4464-a048-cfa9cba9c96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:21.462355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:21.509240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16a158-e9f1-4464-a048-cfa9cba9c96c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:21.641562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feadd4-de77-48c0-969c-74b065c1793a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:21.642346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feadd4-de77-48c0-969c-74b065c1793a service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:21.642640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feadd4-de77-48c0-969c-74b065c1793a service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:21.644029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feadd4-de77-48c0-969c-74b065c1793a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:21.648116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-66feadd4-de77-48c0-969c-74b065c1793a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:21.770783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:21.791155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66feadd4-de77-48c0-969c-74b065c1793a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:21.792806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66feadd4-de77-48c0-969c-74b065c1793a service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:21.814009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 713/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:22.743013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde6f6-aec2-40da-b5d9-6d6fe8562864 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:22.743763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde6f6-aec2-40da-b5d9-6d6fe8562864 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:22.743763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde6f6-aec2-40da-b5d9-6d6fe8562864 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:22.745036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:12:22.750541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24940b-9401-4e6c-b1ed-10a9388d6249 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:22.751251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24940b-9401-4e6c-b1ed-10a9388d6249 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:22.751251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24940b-9401-4e6c-b1ed-10a9388d6249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:22.853860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-dc24940b-9401-4e6c-b1ed-10a9388d6249 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:22.855911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dc24940b-9401-4e6c-b1ed-10a9388d6249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:22.979575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:23.134435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cde6f6-aec2-40da-b5d9-6d6fe8562864 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:23.137042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-93cde6f6-aec2-40da-b5d9-6d6fe8562864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:23.233900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:23.304324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cde6f6-aec2-40da-b5d9-6d6fe8562864 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:23.441866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1b31c-8333-45ca-bc0f-b039070424b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:23.442407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1b31c-8333-45ca-bc0f-b039070424b6 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:23.442688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1b31c-8333-45ca-bc0f-b039070424b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:23.442963 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1b31c-8333-45ca-bc0f-b039070424b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:23.444774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e4e1b31c-8333-45ca-bc0f-b039070424b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:23.508614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:23.554124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1b31c-8333-45ca-bc0f-b039070424b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:23.555814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1b31c-8333-45ca-bc0f-b039070424b6 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:23.576588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 714/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:24.448210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050bbb-567c-47bd-880c-4f1de3176030 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:24.449120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050bbb-567c-47bd-880c-4f1de3176030 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:24.449349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050bbb-567c-47bd-880c-4f1de3176030 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:24.451326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:12:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:24.459378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fde71-aab5-4fbd-9f25-10575d2835d9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:24.460041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fde71-aab5-4fbd-9f25-10575d2835d9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:24.460041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fde71-aab5-4fbd-9f25-10575d2835d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:24.581570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d40fde71-aab5-4fbd-9f25-10575d2835d9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:24.583860 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d40fde71-aab5-4fbd-9f25-10575d2835d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:24.752560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:24.942452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa050bbb-567c-47bd-880c-4f1de3176030 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:24.945308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fa050bbb-567c-47bd-880c-4f1de3176030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:25.025189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:25.142973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa050bbb-567c-47bd-880c-4f1de3176030 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:25.272706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125ee33-9e43-4667-97a6-b8344953acb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:25.273385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125ee33-9e43-4667-97a6-b8344953acb5 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:25.273658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125ee33-9e43-4667-97a6-b8344953acb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:25.273925 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125ee33-9e43-4667-97a6-b8344953acb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:25.275662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6125ee33-9e43-4667-97a6-b8344953acb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:25.283310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:25.391477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125ee33-9e43-4667-97a6-b8344953acb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:25.393087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125ee33-9e43-4667-97a6-b8344953acb5 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:25.415482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 716/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:26.292371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7b96-953f-4045-9a12-b2d16904da94 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:26.293294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7b96-953f-4045-9a12-b2d16904da94 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:26.293294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7b96-953f-4045-9a12-b2d16904da94 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:26.294455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:12:26.300978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642911-52fc-43b7-bdc6-b5846ac112ad None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:26.301525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642911-52fc-43b7-bdc6-b5846ac112ad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:26.301647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642911-52fc-43b7-bdc6-b5846ac112ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:26.395690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-85642911-52fc-43b7-bdc6-b5846ac112ad None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:26.397244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-85642911-52fc-43b7-bdc6-b5846ac112ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:26.473200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:26.514470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:26.686978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c7b96-953f-4045-9a12-b2d16904da94 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:26.689306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ce8c7b96-953f-4045-9a12-b2d16904da94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:26.710007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:26.858153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c7b96-953f-4045-9a12-b2d16904da94 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:26.982093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:27.014648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a26c37-4a4d-4c83-91af-7b1a2563be38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:27.015289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a26c37-4a4d-4c83-91af-7b1a2563be38 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:27.015566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a26c37-4a4d-4c83-91af-7b1a2563be38 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:27.015829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a26c37-4a4d-4c83-91af-7b1a2563be38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:27.017472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-68a26c37-4a4d-4c83-91af-7b1a2563be38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:27.118363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a26c37-4a4d-4c83-91af-7b1a2563be38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:27.120043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a26c37-4a4d-4c83-91af-7b1a2563be38 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:27.140668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 717/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:28.052015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d94220-b4c3-4763-8cd9-67a667b37c0d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:28.052613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d94220-b4c3-4763-8cd9-67a667b37c0d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:28.052613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d94220-b4c3-4763-8cd9-67a667b37c0d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:28.054220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:12:28.059585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c0c4-d8f3-4f61-9f4a-bbb4a53b6234 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:28.060085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c0c4-d8f3-4f61-9f4a-bbb4a53b6234 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:28.060191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c0c4-d8f3-4f61-9f4a-bbb4a53b6234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:28.150450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-36e3c0c4-d8f3-4f61-9f4a-bbb4a53b6234 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:28.151659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-36e3c0c4-d8f3-4f61-9f4a-bbb4a53b6234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:28.209623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:28.274785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:28.431969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d94220-b4c3-4763-8cd9-67a667b37c0d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:28.433460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4d94220-b4c3-4763-8cd9-67a667b37c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:28.619631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d94220-b4c3-4763-8cd9-67a667b37c0d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:28.698418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:28.783503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0dc11-86c8-420b-841e-605f0418b555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:28.784217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0dc11-86c8-420b-841e-605f0418b555 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:28.784541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0dc11-86c8-420b-841e-605f0418b555 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:28.784815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0dc11-86c8-420b-841e-605f0418b555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:28.786788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eea0dc11-86c8-420b-841e-605f0418b555 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:28.906432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0dc11-86c8-420b-841e-605f0418b555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:28.908112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0dc11-86c8-420b-841e-605f0418b555 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:28.937298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 719/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:29.816166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1b641-bb2e-43b3-901b-3797409493ed None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:29.817095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1b641-bb2e-43b3-901b-3797409493ed None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:29.817095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1b641-bb2e-43b3-901b-3797409493ed None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:29.819001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:12:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:29.826656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b846b-096b-448b-9063-4417fa8d0021 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:29.827279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b846b-096b-448b-9063-4417fa8d0021 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:29.827518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b846b-096b-448b-9063-4417fa8d0021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:30.052674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9e3b846b-096b-448b-9063-4417fa8d0021 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:30.054221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9e3b846b-096b-448b-9063-4417fa8d0021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:30.056860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:30.165463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:30.321523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1b641-bb2e-43b3-901b-3797409493ed None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:30.323260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f1a1b641-bb2e-43b3-901b-3797409493ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:30.509019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1b641-bb2e-43b3-901b-3797409493ed None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:30.542800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:30.657405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61b98f-40f6-465e-9117-cb29e08924f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:30.658031 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61b98f-40f6-465e-9117-cb29e08924f8 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:30.658305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61b98f-40f6-465e-9117-cb29e08924f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:30.658569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61b98f-40f6-465e-9117-cb29e08924f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:30.660351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-be61b98f-40f6-465e-9117-cb29e08924f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:30.774256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61b98f-40f6-465e-9117-cb29e08924f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:30.775809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61b98f-40f6-465e-9117-cb29e08924f8 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:30.797043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 720/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:32.656048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5a0b8-b9a1-4c52-a8e9-bbd3be29c2c9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:32.656908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5a0b8-b9a1-4c52-a8e9-bbd3be29c2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:32.656908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5a0b8-b9a1-4c52-a8e9-bbd3be29c2c9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:32.659074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:32.664924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115f856-b2f8-4303-a0e9-9d33bea04f97 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:32.665383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115f856-b2f8-4303-a0e9-9d33bea04f97 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:32.665476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115f856-b2f8-4303-a0e9-9d33bea04f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:32.707542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:32.736761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5115f856-b2f8-4303-a0e9-9d33bea04f97 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:32.737993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5115f856-b2f8-4303-a0e9-9d33bea04f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:32.848720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:33.146226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5a0b8-b9a1-4c52-a8e9-bbd3be29c2c9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:33.147895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-32e5a0b8-b9a1-4c52-a8e9-bbd3be29c2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:33.257472 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5a0b8-b9a1-4c52-a8e9-bbd3be29c2c9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:33.290082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:33.333375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a8c46-2e01-484d-80ac-d26193b49a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:33.334011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8c46-2e01-484d-80ac-d26193b49a7a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:33.334588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8c46-2e01-484d-80ac-d26193b49a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:33.334857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8c46-2e01-484d-80ac-d26193b49a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:33.336505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5a7a8c46-2e01-484d-80ac-d26193b49a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:33.447555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a8c46-2e01-484d-80ac-d26193b49a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:33.448916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a8c46-2e01-484d-80ac-d26193b49a7a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:33.468876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 722/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:35.556519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89408-6786-4c13-82f9-116c789b3a0d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:35.557129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89408-6786-4c13-82f9-116c789b3a0d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:35.557129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89408-6786-4c13-82f9-116c789b3a0d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:35.558419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:35.563903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f07b7-8f28-4de3-bb02-d3dace7b5d14 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:35.564389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f07b7-8f28-4de3-bb02-d3dace7b5d14 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:35.564389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f07b7-8f28-4de3-bb02-d3dace7b5d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:35.637996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-753f07b7-8f28-4de3-bb02-d3dace7b5d14 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:35.639264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-753f07b7-8f28-4de3-bb02-d3dace7b5d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:35.653289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:35.738888 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:36.017906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c89408-6786-4c13-82f9-116c789b3a0d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:36.019511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-99c89408-6786-4c13-82f9-116c789b3a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:36.131064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c89408-6786-4c13-82f9-116c789b3a0d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:36.144019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:36.214274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dff4e-f3f0-4ed6-a647-0edf97aa3c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:36.214915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dff4e-f3f0-4ed6-a647-0edf97aa3c1e service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:36.215193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dff4e-f3f0-4ed6-a647-0edf97aa3c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:36.215500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dff4e-f3f0-4ed6-a647-0edf97aa3c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:36.217147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1a5dff4e-f3f0-4ed6-a647-0edf97aa3c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:36.337721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5dff4e-f3f0-4ed6-a647-0edf97aa3c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:36.339189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5dff4e-f3f0-4ed6-a647-0edf97aa3c1e service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:36.358076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 723/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:38.617018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25100a65-5904-4ccb-91f5-2d9b54acbd26 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:38.617622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25100a65-5904-4ccb-91f5-2d9b54acbd26 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:38.617622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25100a65-5904-4ccb-91f5-2d9b54acbd26 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:38.618889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:12:38.624281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f84c0-faee-44a9-b0cf-bb7c754b2d47 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:38.624718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f84c0-faee-44a9-b0cf-bb7c754b2d47 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:38.624834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f84c0-faee-44a9-b0cf-bb7c754b2d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:38.677503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:38.696210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a20f84c0-faee-44a9-b0cf-bb7c754b2d47 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:38.697435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a20f84c0-faee-44a9-b0cf-bb7c754b2d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:38.815721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:39.123970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25100a65-5904-4ccb-91f5-2d9b54acbd26 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:39.125577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-25100a65-5904-4ccb-91f5-2d9b54acbd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:39.245905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25100a65-5904-4ccb-91f5-2d9b54acbd26 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:39.291452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:39.336300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54bbd3-7d81-4461-a799-33d96f8e996a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:39.337183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54bbd3-7d81-4461-a799-33d96f8e996a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:39.337632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54bbd3-7d81-4461-a799-33d96f8e996a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:39.338058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54bbd3-7d81-4461-a799-33d96f8e996a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:39.340376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6c54bbd3-7d81-4461-a799-33d96f8e996a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:39.475209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54bbd3-7d81-4461-a799-33d96f8e996a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:39.476707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54bbd3-7d81-4461-a799-33d96f8e996a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:39.496822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 725/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:41.616068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b323a-8d59-44d2-9a5d-42ad3ddcdec6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:41.616651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b323a-8d59-44d2-9a5d-42ad3ddcdec6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:41.616754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b323a-8d59-44d2-9a5d-42ad3ddcdec6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:41.618165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:41.623805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c6843-e1ea-47f9-9a83-f08f36e76129 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:41.624222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c6843-e1ea-47f9-9a83-f08f36e76129 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:41.624331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c6843-e1ea-47f9-9a83-f08f36e76129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:41.692575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1d9c6843-e1ea-47f9-9a83-f08f36e76129 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:41.693872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1d9c6843-e1ea-47f9-9a83-f08f36e76129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:41.715725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:41.809283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:42.089202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b323a-8d59-44d2-9a5d-42ad3ddcdec6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:42.090918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c11b323a-8d59-44d2-9a5d-42ad3ddcdec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:42.212826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b323a-8d59-44d2-9a5d-42ad3ddcdec6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:42.235584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:42.305622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901c932-0248-42eb-9667-49f78e827db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:42.306245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901c932-0248-42eb-9667-49f78e827db4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:42.306520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901c932-0248-42eb-9667-49f78e827db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:42.306786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901c932-0248-42eb-9667-49f78e827db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:42.308519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f901c932-0248-42eb-9667-49f78e827db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:42.424877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901c932-0248-42eb-9667-49f78e827db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:42.427010 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901c932-0248-42eb-9667-49f78e827db4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:42.447196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 726/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:44.491453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f7b1e-47ff-4ce9-a792-8b95b420db07 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:44.492067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f7b1e-47ff-4ce9-a792-8b95b420db07 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:44.492067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f7b1e-47ff-4ce9-a792-8b95b420db07 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:44.494129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:44.499985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc630a9-85f8-4afd-856a-f07c2803f282 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:44.500514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc630a9-85f8-4afd-856a-f07c2803f282 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:44.500514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc630a9-85f8-4afd-856a-f07c2803f282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:44.572924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:44.576391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cdc630a9-85f8-4afd-856a-f07c2803f282 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:44.578126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cdc630a9-85f8-4afd-856a-f07c2803f282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:44.680198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:44.960138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f7b1e-47ff-4ce9-a792-8b95b420db07 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:44.961748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-314f7b1e-47ff-4ce9-a792-8b95b420db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:45.070208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f7b1e-47ff-4ce9-a792-8b95b420db07 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:45.127568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:45.160025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98719ac7-739c-4a8b-b09c-62ab23381df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:45.160256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719ac7-739c-4a8b-b09c-62ab23381df7 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:45.160404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719ac7-739c-4a8b-b09c-62ab23381df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:45.160668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719ac7-739c-4a8b-b09c-62ab23381df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:45.162290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-98719ac7-739c-4a8b-b09c-62ab23381df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:45.324964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98719ac7-739c-4a8b-b09c-62ab23381df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:45.326575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98719ac7-739c-4a8b-b09c-62ab23381df7 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:45.346795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 728/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:47.448807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e56c0-7095-4504-b9e3-131125f6a47e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:47.449377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e56c0-7095-4504-b9e3-131125f6a47e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:47.449377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e56c0-7095-4504-b9e3-131125f6a47e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:47.450626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:47.456076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c6f6-562a-47a9-855e-2a4ba316f40b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:47.456514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c6f6-562a-47a9-855e-2a4ba316f40b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:47.456791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c6f6-562a-47a9-855e-2a4ba316f40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:47.536808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d4d2c6f6-562a-47a9-855e-2a4ba316f40b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:47.538045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d4d2c6f6-562a-47a9-855e-2a4ba316f40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:47.571379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:47.648332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:47.944749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e56c0-7095-4504-b9e3-131125f6a47e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:47.946303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1e0e56c0-7095-4504-b9e3-131125f6a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:48.063840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e56c0-7095-4504-b9e3-131125f6a47e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:48.096408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:48.158427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42eb8b-bd39-41ae-8699-c71856a4b4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:48.158975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42eb8b-bd39-41ae-8699-c71856a4b4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:48.159367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42eb8b-bd39-41ae-8699-c71856a4b4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:48.159824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42eb8b-bd39-41ae-8699-c71856a4b4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:48.161641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ed42eb8b-bd39-41ae-8699-c71856a4b4e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:48.287134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42eb8b-bd39-41ae-8699-c71856a4b4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:48.288618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42eb8b-bd39-41ae-8699-c71856a4b4e9 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:48.308226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 729/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:12:50.271140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a556a-a24d-4253-8c06-d79a5d724aad None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:50.271744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a556a-a24d-4253-8c06-d79a5d724aad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:50.271744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a556a-a24d-4253-8c06-d79a5d724aad None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:50.273115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:50.278629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ed2a9-ae4b-48c1-869e-0702c03881d6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:50.279040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ed2a9-ae4b-48c1-869e-0702c03881d6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:50.279169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ed2a9-ae4b-48c1-869e-0702c03881d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:50.354335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-051ed2a9-ae4b-48c1-869e-0702c03881d6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:50.355572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-051ed2a9-ae4b-48c1-869e-0702c03881d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:50.364432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:50.454807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:12:50.734961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a556a-a24d-4253-8c06-d79a5d724aad None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:50.736542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c19a556a-a24d-4253-8c06-d79a5d724aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:50.846136 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a556a-a24d-4253-8c06-d79a5d724aad None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:50.905021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:50.925407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ab402-c9ab-45b5-a334-83e19f069233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:50.926106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ab402-c9ab-45b5-a334-83e19f069233 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:50.926379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ab402-c9ab-45b5-a334-83e19f069233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:50.926646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ab402-c9ab-45b5-a334-83e19f069233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:50.928293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dc2ab402-c9ab-45b5-a334-83e19f069233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:51.040795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ab402-c9ab-45b5-a334-83e19f069233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:51.042147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ab402-c9ab-45b5-a334-83e19f069233 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:51.061214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 731/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:53.041231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf6a3-e23f-4160-aea6-5356514407b2 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:53.041829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf6a3-e23f-4160-aea6-5356514407b2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:53.041829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf6a3-e23f-4160-aea6-5356514407b2 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:53.043173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:53.050528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50f8e-0e9a-46b7-999f-98e44dfa68a2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:53.051064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50f8e-0e9a-46b7-999f-98e44dfa68a2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:53.051064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50f8e-0e9a-46b7-999f-98e44dfa68a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:53.126774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3be50f8e-0e9a-46b7-999f-98e44dfa68a2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:53.128304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3be50f8e-0e9a-46b7-999f-98e44dfa68a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:53.170878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:53.235391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:12:53.518383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf6a3-e23f-4160-aea6-5356514407b2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:53.519963 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f29bf6a3-e23f-4160-aea6-5356514407b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:53.631790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf6a3-e23f-4160-aea6-5356514407b2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:53.669827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:53.709321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ece16-5519-46bc-87c7-698c011fbda4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:53.709975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ece16-5519-46bc-87c7-698c011fbda4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:53.710216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ece16-5519-46bc-87c7-698c011fbda4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:53.710442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ece16-5519-46bc-87c7-698c011fbda4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:53.712127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-216ece16-5519-46bc-87c7-698c011fbda4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:53.828338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ece16-5519-46bc-87c7-698c011fbda4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:53.829684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ece16-5519-46bc-87c7-698c011fbda4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:53.848140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 732/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:55.798366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eea86-853d-4243-ad21-facc0703d111 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:55.799182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eea86-853d-4243-ad21-facc0703d111 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:55.799182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eea86-853d-4243-ad21-facc0703d111 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:55.800326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:55.805939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0f704-ebec-48ef-a247-f6ae83615b31 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:55.806432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0f704-ebec-48ef-a247-f6ae83615b31 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:55.806432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0f704-ebec-48ef-a247-f6ae83615b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:55.880026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-48b0f704-ebec-48ef-a247-f6ae83615b31 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:55.881242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-48b0f704-ebec-48ef-a247-f6ae83615b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:55.900781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:55.983533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:12:56.279281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eea86-853d-4243-ad21-facc0703d111 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:56.280862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bd9eea86-853d-4243-ad21-facc0703d111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:56.399697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eea86-853d-4243-ad21-facc0703d111 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:56.473488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:56.481716 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7096157-df82-4cbf-8340-07b68b6c5940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:56.482503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7096157-df82-4cbf-8340-07b68b6c5940 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:56.482780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7096157-df82-4cbf-8340-07b68b6c5940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:56.483045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7096157-df82-4cbf-8340-07b68b6c5940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:56.484740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e7096157-df82-4cbf-8340-07b68b6c5940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:56.599016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7096157-df82-4cbf-8340-07b68b6c5940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:56.600400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7096157-df82-4cbf-8340-07b68b6c5940 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:56.619384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 734/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:12:58.625402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5e042-115f-4a7d-b124-d09f898196ba None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:58.626025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5e042-115f-4a7d-b124-d09f898196ba None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:58.626025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5e042-115f-4a7d-b124-d09f898196ba None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:58.627972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:12:58.635880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dc295-b5df-4909-bce2-e707d62fe4a7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:58.636324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dc295-b5df-4909-bce2-e707d62fe4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:58.636435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dc295-b5df-4909-bce2-e707d62fe4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:58.706857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ab3dc295-b5df-4909-bce2-e707d62fe4a7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:12:58.708100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ab3dc295-b5df-4909-bce2-e707d62fe4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:58.757945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:58.808678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:12:59.097762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5e042-115f-4a7d-b124-d09f898196ba None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:12:59.099408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-30a5e042-115f-4a7d-b124-d09f898196ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:59.213483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5e042-115f-4a7d-b124-d09f898196ba None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:12:59.268199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:12:59.297041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5b830-c2d9-46be-8b35-85bc7f0d3151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:12:59.297747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5b830-c2d9-46be-8b35-85bc7f0d3151 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:12:59.298100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5b830-c2d9-46be-8b35-85bc7f0d3151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:12:59.298383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5b830-c2d9-46be-8b35-85bc7f0d3151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:12:59.300174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-82c5b830-c2d9-46be-8b35-85bc7f0d3151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:12:59.413903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5b830-c2d9-46be-8b35-85bc7f0d3151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:12:59.415580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5b830-c2d9-46be-8b35-85bc7f0d3151 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:12:59.437204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 735/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:13:08.806675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=22 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:09.433822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=66 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:19.225046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12e1c-efbb-45e8-b464-6caa884d47b9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:19.225899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12e1c-efbb-45e8-b464-6caa884d47b9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:19.225899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12e1c-efbb-45e8-b464-6caa884d47b9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:19.227534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 736/1471] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:13:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:13:19.233748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778e57a-dee0-4422-a9d7-518b044c78f4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:19.234402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778e57a-dee0-4422-a9d7-518b044c78f4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:19.234402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778e57a-dee0-4422-a9d7-518b044c78f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:19.323717 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9778e57a-dee0-4422-a9d7-518b044c78f4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:13:19.325025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9778e57a-dee0-4422-a9d7-518b044c78f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:19.456316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 736/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:13:19.818987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512bb80-1f93-42da-873b-0106f5aaa0c2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:19.818987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512bb80-1f93-42da-873b-0106f5aaa0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:19.818987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512bb80-1f93-42da-873b-0106f5aaa0c2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:19.818987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 737/1473] 10.222.0.27 () {54 vars in 967 bytes} [Wed Aug 31 12:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:13:19.827121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2353b1-7528-4408-a175-cbcbb54d52f4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:19.827940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2353b1-7528-4408-a175-cbcbb54d52f4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:19.828847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2353b1-7528-4408-a175-cbcbb54d52f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:19.968736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7c2353b1-7528-4408-a175-cbcbb54d52f4 None None] MFA Rules not processed for user `eef756349e5844e59d7b20072f63c41c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:13:19.970101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7c2353b1-7528-4408-a175-cbcbb54d52f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:19.997439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:20.083539 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 737/1474] 10.222.0.27 () {58 vars in 1064 bytes} [Wed Aug 31 12:13:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:13:20.099379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512bb80-1f93-42da-873b-0106f5aaa0c2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:13:20.102196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4512bb80-1f93-42da-873b-0106f5aaa0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:20.268540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512bb80-1f93-42da-873b-0106f5aaa0c2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:13:20.357566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a997cc-d37a-4b96-ae56-45ffdc1d34da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:13:20.358127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a997cc-d37a-4b96-ae56-45ffdc1d34da service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:20.358468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a997cc-d37a-4b96-ae56-45ffdc1d34da service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:20.358794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a997cc-d37a-4b96-ae56-45ffdc1d34da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:20.360571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-73a997cc-d37a-4b96-ae56-45ffdc1d34da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:20.430640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:20.522432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a997cc-d37a-4b96-ae56-45ffdc1d34da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:13:20.525078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a997cc-d37a-4b96-ae56-45ffdc1d34da service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:13:20.561720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 738/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:13:22.679610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ce262-5cc7-4869-92f7-4a5f51c924c4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:22.680427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ce262-5cc7-4869-92f7-4a5f51c924c4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:22.680427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ce262-5cc7-4869-92f7-4a5f51c924c4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:22.681952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 738/1476] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:13:22.688780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a877e9-9ed5-4ff0-95ba-1946dcf03f96 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:22.689363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a877e9-9ed5-4ff0-95ba-1946dcf03f96 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:22.689507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a877e9-9ed5-4ff0-95ba-1946dcf03f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:22.785554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-80a877e9-9ed5-4ff0-95ba-1946dcf03f96 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:13:22.786757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-80a877e9-9ed5-4ff0-95ba-1946dcf03f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:22.897621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 739/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:13:23.182295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ce262-5cc7-4869-92f7-4a5f51c924c4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:13:23.184013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3e1ce262-5cc7-4869-92f7-4a5f51c924c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:23.324241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ce262-5cc7-4869-92f7-4a5f51c924c4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:13:23.389529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:23.446215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c055-7f77-434d-a9a7-0b5bd03a153a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:13:23.446767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c055-7f77-434d-a9a7-0b5bd03a153a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:23.447042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c055-7f77-434d-a9a7-0b5bd03a153a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:23.447352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c055-7f77-434d-a9a7-0b5bd03a153a admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:23.448014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416c055-7f77-434d-a9a7-0b5bd03a153a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:13:23.449959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-7416c055-7f77-434d-a9a7-0b5bd03a153a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:13:23.449959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Aug 31 12:13:23.449959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: Aug 31 12:13:23.453700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416c055-7f77-434d-a9a7-0b5bd03a153a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:13:23.491754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 739/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 31 12:13:23 2022] GET /identity/v3/projects => generated 2553 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:13:25.490872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51bfa3-4abd-4ea7-87d9-807c99a9c0b5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:25.491669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51bfa3-4abd-4ea7-87d9-807c99a9c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:25.491669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51bfa3-4abd-4ea7-87d9-807c99a9c0b5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:25.493046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 740/1479] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:13:25.499142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9667c-ea2c-47af-9e37-2e88154227c3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:25.499779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9667c-ea2c-47af-9e37-2e88154227c3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:25.499779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9667c-ea2c-47af-9e37-2e88154227c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:25.572166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e0e9667c-ea2c-47af-9e37-2e88154227c3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:13:25.573443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e0e9667c-ea2c-47af-9e37-2e88154227c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:25.684826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 740/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:13:25.973711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51bfa3-4abd-4ea7-87d9-807c99a9c0b5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:13:25.975263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ed51bfa3-4abd-4ea7-87d9-807c99a9c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:25.986310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:26.143376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51bfa3-4abd-4ea7-87d9-807c99a9c0b5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:13:26.232543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:26.271253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98217a7-16b5-4d3f-b579-2182ca143cdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:13:26.271876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98217a7-16b5-4d3f-b579-2182ca143cdf service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:26.272154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98217a7-16b5-4d3f-b579-2182ca143cdf service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:26.272539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98217a7-16b5-4d3f-b579-2182ca143cdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:26.274735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d98217a7-16b5-4d3f-b579-2182ca143cdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:26.380423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98217a7-16b5-4d3f-b579-2182ca143cdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:13:26.382035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98217a7-16b5-4d3f-b579-2182ca143cdf service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:13:26.401895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 741/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:13:28.625469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7741a-22c1-420b-8578-169b458e6ef1 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:28.626084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7741a-22c1-420b-8578-169b458e6ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:28.626084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7741a-22c1-420b-8578-169b458e6ef1 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:28.627346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 741/1482] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:13:28.632937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc50c-30f4-4da3-b60f-d211d41e8c18 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:28.633446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc50c-30f4-4da3-b60f-d211d41e8c18 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:28.633446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc50c-30f4-4da3-b60f-d211d41e8c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:28.706171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bdccc50c-30f4-4da3-b60f-d211d41e8c18 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:13:28.707417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bdccc50c-30f4-4da3-b60f-d211d41e8c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:28.756003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:28.808960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 742/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:13:29.108556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7741a-22c1-420b-8578-169b458e6ef1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:13:29.110139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0ac7741a-22c1-420b-8578-169b458e6ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:29.134883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:29.227470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7741a-22c1-420b-8578-169b458e6ef1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:13:29.319321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb75bf-de1a-4ddb-802d-8d818581f2ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:13:29.319948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb75bf-de1a-4ddb-802d-8d818581f2ab service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:29.320226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb75bf-de1a-4ddb-802d-8d818581f2ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:29.320494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb75bf-de1a-4ddb-802d-8d818581f2ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:29.322420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d0eb75bf-de1a-4ddb-802d-8d818581f2ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:29.408084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:29.448412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb75bf-de1a-4ddb-802d-8d818581f2ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:13:29.450045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb75bf-de1a-4ddb-802d-8d818581f2ab service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:13:29.470464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 742/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:13:31.464405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006182f-3e17-402d-a295-3af2a7ff594e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:31.464985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006182f-3e17-402d-a295-3af2a7ff594e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:31.465142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006182f-3e17-402d-a295-3af2a7ff594e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:31.467027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 743/1485] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:13:31.472726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac069d-bfe9-40a5-a8dc-e8acbf24123a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:31.473171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac069d-bfe9-40a5-a8dc-e8acbf24123a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:31.473268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac069d-bfe9-40a5-a8dc-e8acbf24123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:31.543097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f8ac069d-bfe9-40a5-a8dc-e8acbf24123a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:13:31.544379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f8ac069d-bfe9-40a5-a8dc-e8acbf24123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:31.648064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 743/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:13:31.940544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006182f-3e17-402d-a295-3af2a7ff594e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:13:31.942127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9006182f-3e17-402d-a295-3af2a7ff594e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:32.065470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006182f-3e17-402d-a295-3af2a7ff594e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:13:32.144987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:32.146808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcf028-4645-44a8-b08d-6e08358bda6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:13:32.147444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf028-4645-44a8-b08d-6e08358bda6a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:32.147731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf028-4645-44a8-b08d-6e08358bda6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:32.148010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf028-4645-44a8-b08d-6e08358bda6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:32.150221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-28bcf028-4645-44a8-b08d-6e08358bda6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:32.269365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcf028-4645-44a8-b08d-6e08358bda6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:13:32.270802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcf028-4645-44a8-b08d-6e08358bda6a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:13:32.290040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 744/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:13:34.458161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6fe7c-54af-43ca-aacb-c2887cb82327 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:34.458876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6fe7c-54af-43ca-aacb-c2887cb82327 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:34.458876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6fe7c-54af-43ca-aacb-c2887cb82327 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:34.460185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 744/1488] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:13:34.466778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5891a01-7ce7-4f3b-ac2b-159c8fb8809c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:34.467370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5891a01-7ce7-4f3b-ac2b-159c8fb8809c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:34.467370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5891a01-7ce7-4f3b-ac2b-159c8fb8809c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:34.541535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c5891a01-7ce7-4f3b-ac2b-159c8fb8809c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:13:34.543667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c5891a01-7ce7-4f3b-ac2b-159c8fb8809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:34.602736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:34.652239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 745/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:13:34.956875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6fe7c-54af-43ca-aacb-c2887cb82327 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:13:34.958436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ecb6fe7c-54af-43ca-aacb-c2887cb82327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:34.999852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:35.074771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6fe7c-54af-43ca-aacb-c2887cb82327 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:13:35.153789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b640ced-680b-4861-a08a-33dd97c1b114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:13:35.154309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b640ced-680b-4861-a08a-33dd97c1b114 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:35.154590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b640ced-680b-4861-a08a-33dd97c1b114 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:35.154860 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b640ced-680b-4861-a08a-33dd97c1b114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:35.156554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6b640ced-680b-4861-a08a-33dd97c1b114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:35.245955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:35.278779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b640ced-680b-4861-a08a-33dd97c1b114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:13:35.280294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b640ced-680b-4861-a08a-33dd97c1b114 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:13:35.299630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 745/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:13:37.535123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832430d-c801-4a1c-aea5-712532eed082 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:37.535721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832430d-c801-4a1c-aea5-712532eed082 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:37.535721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832430d-c801-4a1c-aea5-712532eed082 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:37.536966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 746/1491] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 31 12:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:13:37.542515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161d35-8f99-4f14-b4d3-3309eb0cd0ee None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:37.543024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161d35-8f99-4f14-b4d3-3309eb0cd0ee None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:37.543024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161d35-8f99-4f14-b4d3-3309eb0cd0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:37.614475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d9161d35-8f99-4f14-b4d3-3309eb0cd0ee None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:13:37.615769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9161d35-8f99-4f14-b4d3-3309eb0cd0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:37.731504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 746/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 31 12:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:13:38.019148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832430d-c801-4a1c-aea5-712532eed082 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:13:38.020735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a832430d-c801-4a1c-aea5-712532eed082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:38.135759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832430d-c801-4a1c-aea5-712532eed082 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:13:38.216611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1849bd-0052-4ab4-be8f-d99ecc61681a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:13:38.217447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1849bd-0052-4ab4-be8f-d99ecc61681a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:13:38.218183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1849bd-0052-4ab4-be8f-d99ecc61681a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:13:38.218499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1849bd-0052-4ab4-be8f-d99ecc61681a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:13:38.220214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-df1849bd-0052-4ab4-be8f-d99ecc61681a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:13:38.230575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:38.342323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1849bd-0052-4ab4-be8f-d99ecc61681a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:13:38.343801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1849bd-0052-4ab4-be8f-d99ecc61681a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:13:38.363758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 747/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:13:47.719857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=82 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:13:48.360942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=50 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:54.070540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffb47a-4a2d-47a0-881f-1e2d2df0c3b8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:54.071204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffb47a-4a2d-47a0-881f-1e2d2df0c3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:54.071204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffb47a-4a2d-47a0-881f-1e2d2df0c3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:54.161228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-77ffb47a-4a2d-47a0-881f-1e2d2df0c3b8 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:14:54.162355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-77ffb47a-4a2d-47a0-881f-1e2d2df0c3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:54.275966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 747/1494] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:14:54.285267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951e1b-3653-4d15-98ee-0e17d6fd16cb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:54.286124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951e1b-3653-4d15-98ee-0e17d6fd16cb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:54.286124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951e1b-3653-4d15-98ee-0e17d6fd16cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:54.380117 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2f951e1b-3653-4d15-98ee-0e17d6fd16cb None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:14:54.381360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2f951e1b-3653-4d15-98ee-0e17d6fd16cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:54.490858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 748/1495] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:14:54.496005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffb47a-4a2d-47a0-881f-1e2d2df0c3b8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:54.497292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-77ffb47a-4a2d-47a0-881f-1e2d2df0c3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:54.510103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffb47a-4a2d-47a0-881f-1e2d2df0c3b8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:54.583383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:54.748116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:54.748632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:54.748901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:54.749163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:54.749815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:14:54.750111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:54.751284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:14:54.751284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Aug 31 12:14:54.751284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: Aug 31 12:14:54.752183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:54.771876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 748/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:14:54.780920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f951e1b-3653-4d15-98ee-0e17d6fd16cb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:54.791958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f951e1b-3653-4d15-98ee-0e17d6fd16cb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:54.859615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:54.901138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:54.901763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:54.902037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:54.902310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:54.903000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:54.904894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:14:54.904894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Aug 31 12:14:54.904894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: Aug 31 12:14:54.906405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:54.990525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 749/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:14:54 2022] POST /identity/v3/projects => generated 379 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:14:54.996764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:54.998465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:55.040072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e02b3-9fa2-49f7-a390-cc357805326a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:55.143832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:55.144642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:55.144797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:55.144951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:55.145587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:55.146631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:14:55.146631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Aug 31 12:14:55.146631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: Aug 31 12:14:55.147886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:55.195456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 749/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:14:54 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:14:55.200109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:55.201856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:55.279446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835a3fb-5d15-4a0c-9c48-d1c41419ac26 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:55.403553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:55.415448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:55.416071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:55.416600 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:55.416877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:55.417540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:55.419698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:14:55.419698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Aug 31 12:14:55.419698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: Aug 31 12:14:55.421461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:55.445183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 750/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:14:55.449401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:55.450962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:55.464679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85fe36-e5ee-4b29-a03e-ddbba27d5ea2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:55.497490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:55.594220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1b393-80f7-4a20-a463-2e95aba1613c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:55.595667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1b393-80f7-4a20-a463-2e95aba1613c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:55.595961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1b393-80f7-4a20-a463-2e95aba1613c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:55.596234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1b393-80f7-4a20-a463-2e95aba1613c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:55.596890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1b393-80f7-4a20-a463-2e95aba1613c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:55.597926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-7fe1b393-80f7-4a20-a463-2e95aba1613c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:14:55.597926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Aug 31 12:14:55.597926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: Aug 31 12:14:55.598822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1b393-80f7-4a20-a463-2e95aba1613c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:55.612584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 750/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:14:55.617360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:55.618924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:55.674182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe08890-0387-49c8-a3ff-e805b66980d5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:55.786240 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49119b64-aded-490a-8f95-494b94ddd623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:55.786935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49119b64-aded-490a-8f95-494b94ddd623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:55.787208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49119b64-aded-490a-8f95-494b94ddd623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:55.787500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49119b64-aded-490a-8f95-494b94ddd623 admin admin] PATH_INFO: `/v3/projects/6ba35674239e4093954ae83c89facd0f/users/51d4bf04b24045c7a7cd251b06d085e3/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:55.811572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49119b64-aded-490a-8f95-494b94ddd623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba35674239e4093954ae83c89facd0f, user_id=51d4bf04b24045c7a7cd251b06d085e3, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:55.812638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-49119b64-aded-490a-8f95-494b94ddd623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:14:55.812638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Aug 31 12:14:55.812638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: Aug 31 12:14:55.814857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49119b64-aded-490a-8f95-494b94ddd623 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:55.844038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 751/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:14:55 2022] PUT /identity/v3/projects/6ba35674239e4093954ae83c89facd0f/users/51d4bf04b24045c7a7cd251b06d085e3/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:14:55.857981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000736d-c29c-45b2-83ca-26b7418254b6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:55.858213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000736d-c29c-45b2-83ca-26b7418254b6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:55.858461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000736d-c29c-45b2-83ca-26b7418254b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:55.930744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3000736d-c29c-45b2-83ca-26b7418254b6 None None] MFA Rules not processed for user `51d4bf04b24045c7a7cd251b06d085e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:14:55.932097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3000736d-c29c-45b2-83ca-26b7418254b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:56.057451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 751/1502] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:14:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:14:56.073283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424d8ea-e814-466b-8403-633106ee542f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:56.073737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424d8ea-e814-466b-8403-633106ee542f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:56.074074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424d8ea-e814-466b-8403-633106ee542f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:56.076027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 752/1503] 10.222.0.27 () {54 vars in 964 bytes} [Wed Aug 31 12:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:14:56.084154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaef7a-9327-4467-a309-09b990395182 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:56.084426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaef7a-9327-4467-a309-09b990395182 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:56.084702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaef7a-9327-4467-a309-09b990395182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:56.130863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:56.182377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4ceaef7a-9327-4467-a309-09b990395182 None None] MFA Rules not processed for user `db3393d3cb0e4bbcb21e403c431bf61b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:14:56.183994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4ceaef7a-9327-4467-a309-09b990395182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:56.275462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 752/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Aug 31 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:14:56.286510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424d8ea-e814-466b-8403-633106ee542f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:56.288090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6424d8ea-e814-466b-8403-633106ee542f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:56.342803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:56.402368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424d8ea-e814-466b-8403-633106ee542f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:56.481293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:56.482025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:56.482176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:56.482454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:56.484200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:56.606488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:56.609426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:56.615969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:56.633203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 753/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:14:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:14:56.683146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c91ff-c56e-4d33-a4a3-596c89dcbba7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:56.683507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c91ff-c56e-4d33-a4a3-596c89dcbba7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:56.683806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c91ff-c56e-4d33-a4a3-596c89dcbba7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:56.685299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 753/1506] 10.222.0.27 () {52 vars in 830 bytes} [Wed Aug 31 12:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:14:56.689977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:56.691511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:56.804668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bb6cb-7296-4f78-9c7e-54a245ed20eb service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:56.916361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:56.916927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:56.917206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:56.917476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] PATH_INFO: `/v3/users/51d4bf04b24045c7a7cd251b06d085e3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:56.932059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d4bf04b24045c7a7cd251b06d085e3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:56.933130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:14:56.933130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Aug 31 12:14:56.933130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: Aug 31 12:14:56.934373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:56.946036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:57.011090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d4bf04b24045c7a7cd251b06d085e3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:57.011686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d4bf04b24045c7a7cd251b06d085e3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:57.024813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d4bf04b24045c7a7cd251b06d085e3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:57.025477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d4bf04b24045c7a7cd251b06d085e3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:57.052388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d4bf04b24045c7a7cd251b06d085e3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:57.073707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 754/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:14:56 2022] DELETE /identity/v3/users/51d4bf04b24045c7a7cd251b06d085e3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:14:57.078339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f9d2-e320-420a-a52e-a7d8d7fc3af6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:57.078756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f9d2-e320-420a-a52e-a7d8d7fc3af6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:57.079047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f9d2-e320-420a-a52e-a7d8d7fc3af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:57.147055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6136f9d2-e320-420a-a52e-a7d8d7fc3af6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:14:57.149288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6136f9d2-e320-420a-a52e-a7d8d7fc3af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:57.276229 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:57.289372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 754/1508] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:14:57.299407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:57.300686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:57.438468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec64b0f-54b8-4cea-8901-4f585b1f1719 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:57.503201 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:14:57.526975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78340578-d997-4f75-b208-664c2e639d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:57.528177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78340578-d997-4f75-b208-664c2e639d8e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:57.528457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78340578-d997-4f75-b208-664c2e639d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:57.529244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78340578-d997-4f75-b208-664c2e639d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:57.533584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-78340578-d997-4f75-b208-664c2e639d8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:57.659639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78340578-d997-4f75-b208-664c2e639d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:57.661131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78340578-d997-4f75-b208-664c2e639d8e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:57.686102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 755/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:14:58.055648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136f9d2-e320-420a-a52e-a7d8d7fc3af6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:14:58.057398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6136f9d2-e320-420a-a52e-a7d8d7fc3af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:14:58.070075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136f9d2-e320-420a-a52e-a7d8d7fc3af6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:14:58.192597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:14:58.193265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:14:58.193567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:14:58.193812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] PATH_INFO: `/v3/projects/6ba35674239e4093954ae83c89facd0f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:14:58.205359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba35674239e4093954ae83c89facd0f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:14:58.206485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:14:58.206485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Aug 31 12:14:58.206485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: Aug 31 12:14:58.207739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:14:58.273802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba35674239e4093954ae83c89facd0f', 'request_id': 'req-37dac73e-4873-4bac-b85e-992506577494'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:58.280091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba35674239e4093954ae83c89facd0f', 'request_id': 'req-37dac73e-4873-4bac-b85e-992506577494'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:58.293400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba35674239e4093954ae83c89facd0f', 'request_id': 'req-37dac73e-4873-4bac-b85e-992506577494'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:58.331647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] The token cache is being invalidate because project 6ba35674239e4093954ae83c89facd0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:14:58.331894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:58.332276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-37dac73e-4873-4bac-b85e-992506577494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:14:58.333754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 755/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:14:58 2022] DELETE /identity/v3/projects/6ba35674239e4093954ae83c89facd0f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:15:07.679977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=62 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:15:08.328882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:16:29.418168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4947a-23ff-4d4c-83b6-79d394dd462a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:29.418872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4947a-23ff-4d4c-83b6-79d394dd462a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:29.418872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4947a-23ff-4d4c-83b6-79d394dd462a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:29.420111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:29.427855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a76ea0-54c5-450d-b3c6-a3d01ff893d3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:29.428417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a76ea0-54c5-450d-b3c6-a3d01ff893d3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:29.428417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a76ea0-54c5-450d-b3c6-a3d01ff893d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:29.530733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-95a76ea0-54c5-450d-b3c6-a3d01ff893d3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:29.531925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95a76ea0-54c5-450d-b3c6-a3d01ff893d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:29.649134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:16:29.801507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4947a-23ff-4d4c-83b6-79d394dd462a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:29.804022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5fa4947a-23ff-4d4c-83b6-79d394dd462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:29.932054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4947a-23ff-4d4c-83b6-79d394dd462a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:29.938891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fb7d9-2f7e-45b3-bb55-33693673fcc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:29.939547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fb7d9-2f7e-45b3-bb55-33693673fcc1 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:29.939826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fb7d9-2f7e-45b3-bb55-33693673fcc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:29.940100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fb7d9-2f7e-45b3-bb55-33693673fcc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:29.949698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fb7d9-2f7e-45b3-bb55-33693673fcc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:29.951363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fb7d9-2f7e-45b3-bb55-33693673fcc1 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:29.961454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 757/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:16:30.801370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7460-0d37-4bcb-9eac-4f9ed7e4bbbc None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:30.801991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7460-0d37-4bcb-9eac-4f9ed7e4bbbc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:30.801991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7460-0d37-4bcb-9eac-4f9ed7e4bbbc None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:30.803269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:30.808413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507520a-bc5b-4383-8108-0a56c1f55c55 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:30.808877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507520a-bc5b-4383-8108-0a56c1f55c55 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:30.808992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507520a-bc5b-4383-8108-0a56c1f55c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:30.830428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9507520a-bc5b-4383-8108-0a56c1f55c55 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:30.831722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9507520a-bc5b-4383-8108-0a56c1f55c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:30.840737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:16:30.994678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e7460-0d37-4bcb-9eac-4f9ed7e4bbbc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:31.011753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e7460-0d37-4bcb-9eac-4f9ed7e4bbbc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:31.017819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b50a7-ea87-4ee3-996f-f99e078d0a71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:31.018446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b50a7-ea87-4ee3-996f-f99e078d0a71 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:31.018712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b50a7-ea87-4ee3-996f-f99e078d0a71 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:31.018986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b50a7-ea87-4ee3-996f-f99e078d0a71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:31.030211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b50a7-ea87-4ee3-996f-f99e078d0a71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:31.031958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b50a7-ea87-4ee3-996f-f99e078d0a71 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:31.043109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 758/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:16:31.592837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df929-7caf-4573-9c01-47eae6fc937a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:31.593073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df929-7caf-4573-9c01-47eae6fc937a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:31.593249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df929-7caf-4573-9c01-47eae6fc937a None None] PATH_INFO: `/v3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:31.594575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 759/1517] 10.222.0.20 () {54 vars in 925 bytes} [Wed Aug 31 12:16:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:16:31.610038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579ff1c-c1ad-495d-bbef-a29417961b76 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:31.610326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579ff1c-c1ad-495d-bbef-a29417961b76 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:31.610594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579ff1c-c1ad-495d-bbef-a29417961b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:31.673165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c579ff1c-c1ad-495d-bbef-a29417961b76 None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:31.674379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c579ff1c-c1ad-495d-bbef-a29417961b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:31.692223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:16:31.749412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 759/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 31 12:16:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:16:35.666362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90210ea-ab16-4fa9-bae7-175d86c308aa None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:35.666927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90210ea-ab16-4fa9-bae7-175d86c308aa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:35.666927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90210ea-ab16-4fa9-bae7-175d86c308aa None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:35.668500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 760/1519] 10.222.0.27 () {54 vars in 961 bytes} [Wed Aug 31 12:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:35.674088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b5df-8fed-419a-9331-a31fbb6c1605 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:35.674613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b5df-8fed-419a-9331-a31fbb6c1605 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:35.674613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b5df-8fed-419a-9331-a31fbb6c1605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:35.698087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9215b5df-8fed-419a-9331-a31fbb6c1605 None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:35.699342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9215b5df-8fed-419a-9331-a31fbb6c1605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:35.708967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 760/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Aug 31 12:16:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:16:35.717178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90210ea-ab16-4fa9-bae7-175d86c308aa None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:35.732250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90210ea-ab16-4fa9-bae7-175d86c308aa None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:35.737938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23afb5-28e6-49b4-a3ad-841adc090812 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:35.738537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23afb5-28e6-49b4-a3ad-841adc090812 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:35.738814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23afb5-28e6-49b4-a3ad-841adc090812 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:35.739082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23afb5-28e6-49b4-a3ad-841adc090812 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:35.748348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23afb5-28e6-49b4-a3ad-841adc090812 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:35.749987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23afb5-28e6-49b4-a3ad-841adc090812 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:35.761141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 761/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:16:42.379770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf927d8-1813-40eb-999f-3e69b34c43c6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:42.380428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf927d8-1813-40eb-999f-3e69b34c43c6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:42.380428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf927d8-1813-40eb-999f-3e69b34c43c6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:42.381732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:42.387768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384fdf-b455-4336-b974-79d62a1e569b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:42.388386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384fdf-b455-4336-b974-79d62a1e569b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:42.388386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384fdf-b455-4336-b974-79d62a1e569b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:42.409385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1f384fdf-b455-4336-b974-79d62a1e569b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:42.410637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1f384fdf-b455-4336-b974-79d62a1e569b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:42.419073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:16:42.577206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf927d8-1813-40eb-999f-3e69b34c43c6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:42.593230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf927d8-1813-40eb-999f-3e69b34c43c6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:42.599156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92d92d-86f6-4313-bdcb-8d66a78e6d3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:42.599819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92d92d-86f6-4313-bdcb-8d66a78e6d3d service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:42.600098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92d92d-86f6-4313-bdcb-8d66a78e6d3d service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:42.600368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92d92d-86f6-4313-bdcb-8d66a78e6d3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:42.609932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92d92d-86f6-4313-bdcb-8d66a78e6d3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:42.611607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92d92d-86f6-4313-bdcb-8d66a78e6d3d service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:42.622470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 762/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:16:43.474857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f1be4-5f59-40fa-a836-1845361cdeed None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:43.475535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f1be4-5f59-40fa-a836-1845361cdeed None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:43.475535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f1be4-5f59-40fa-a836-1845361cdeed None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:43.476885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 12:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:43.482267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c22a9-3215-4ce8-9f1a-242c432c9248 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:43.482770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c22a9-3215-4ce8-9f1a-242c432c9248 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:43.482879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c22a9-3215-4ce8-9f1a-242c432c9248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:43.507969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8a1c22a9-3215-4ce8-9f1a-242c432c9248 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:43.509399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a1c22a9-3215-4ce8-9f1a-242c432c9248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:43.519639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:16:43.679798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f1be4-5f59-40fa-a836-1845361cdeed None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:43.695196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f1be4-5f59-40fa-a836-1845361cdeed None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:43.701377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd152a-49a3-43de-bc73-28496b022e87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:43.702000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd152a-49a3-43de-bc73-28496b022e87 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:43.702278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd152a-49a3-43de-bc73-28496b022e87 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:43.702555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd152a-49a3-43de-bc73-28496b022e87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:43.711603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd152a-49a3-43de-bc73-28496b022e87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:43.713726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd152a-49a3-43de-bc73-28496b022e87 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:43.724511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 764/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:16:47.701131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afd933-255d-4fe5-ab13-bf627c36c124 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:47.701726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afd933-255d-4fe5-ab13-bf627c36c124 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:47.701726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afd933-255d-4fe5-ab13-bf627c36c124 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:47.702917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:47.708515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec144e-e582-4bbd-835c-e8f5297c06f9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:47.709004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec144e-e582-4bbd-835c-e8f5297c06f9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:47.709004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec144e-e582-4bbd-835c-e8f5297c06f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:47.730201 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c4ec144e-e582-4bbd-835c-e8f5297c06f9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:47.731466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4ec144e-e582-4bbd-835c-e8f5297c06f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:47.740058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:16:48.026117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afd933-255d-4fe5-ab13-bf627c36c124 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:48.028885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-04afd933-255d-4fe5-ab13-bf627c36c124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:48.134000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afd933-255d-4fe5-ab13-bf627c36c124 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:48.140923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c363fc4-7c64-4177-8227-80c1839e5dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:48.141822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c363fc4-7c64-4177-8227-80c1839e5dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:48.142236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c363fc4-7c64-4177-8227-80c1839e5dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:48.142653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c363fc4-7c64-4177-8227-80c1839e5dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:48.156342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c363fc4-7c64-4177-8227-80c1839e5dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:48.157809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c363fc4-7c64-4177-8227-80c1839e5dc4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:48.169057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 765/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:16:50.069778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8e11a-9eef-487b-bd2f-f7090c6c1876 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:50.070407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8e11a-9eef-487b-bd2f-f7090c6c1876 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:50.070407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8e11a-9eef-487b-bd2f-f7090c6c1876 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:50.071845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:50.078604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dd953-877e-49df-8a03-0fb04e0b0273 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:50.079459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dd953-877e-49df-8a03-0fb04e0b0273 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:50.079651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dd953-877e-49df-8a03-0fb04e0b0273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:50.103515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-070dd953-877e-49df-8a03-0fb04e0b0273 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:50.104776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-070dd953-877e-49df-8a03-0fb04e0b0273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:50.114375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:16:50.405191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8e11a-9eef-487b-bd2f-f7090c6c1876 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:50.416755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8e11a-9eef-487b-bd2f-f7090c6c1876 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:50.420482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163d8f3-193a-44cc-a35c-fc2cd8692f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:50.421113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d8f3-193a-44cc-a35c-fc2cd8692f70 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:50.421387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d8f3-193a-44cc-a35c-fc2cd8692f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:50.421656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d8f3-193a-44cc-a35c-fc2cd8692f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:50.431254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163d8f3-193a-44cc-a35c-fc2cd8692f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:50.432676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163d8f3-193a-44cc-a35c-fc2cd8692f70 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:50.442887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 767/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:16:54.099090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883a14-c4b3-4e7e-84e2-f02fe813ea0b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:54.099716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883a14-c4b3-4e7e-84e2-f02fe813ea0b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:54.099716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883a14-c4b3-4e7e-84e2-f02fe813ea0b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:54.101014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:54.106568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b13b-ac18-4f5c-a89e-c7021cff561a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:54.107018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b13b-ac18-4f5c-a89e-c7021cff561a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:54.107135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b13b-ac18-4f5c-a89e-c7021cff561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:54.130127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1c44b13b-ac18-4f5c-a89e-c7021cff561a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:54.131384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1c44b13b-ac18-4f5c-a89e-c7021cff561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:54.140896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:16:54.707291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a883a14-c4b3-4e7e-84e2-f02fe813ea0b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:54.709029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a883a14-c4b3-4e7e-84e2-f02fe813ea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:54.784851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:16:54.800421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a883a14-c4b3-4e7e-84e2-f02fe813ea0b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:54.805035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ca8f3-849a-4950-bfbe-c73c2bfaec44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:54.805687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ca8f3-849a-4950-bfbe-c73c2bfaec44 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:54.805958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ca8f3-849a-4950-bfbe-c73c2bfaec44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:54.806326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ca8f3-849a-4950-bfbe-c73c2bfaec44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:54.816681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ca8f3-849a-4950-bfbe-c73c2bfaec44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:54.818219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ca8f3-849a-4950-bfbe-c73c2bfaec44 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:54.828723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 768/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:16:56.884502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83f0e-30ae-4f0b-bd6a-10be82c63b19 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:56.885078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83f0e-30ae-4f0b-bd6a-10be82c63b19 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:56.885078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83f0e-30ae-4f0b-bd6a-10be82c63b19 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:56.886392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 12:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:16:56.891849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4a46-47fb-4089-973b-e6bc2abc062e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:56.892368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4a46-47fb-4089-973b-e6bc2abc062e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:56.892368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4a46-47fb-4089-973b-e6bc2abc062e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:56.912962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f6cd4a46-47fb-4089-973b-e6bc2abc062e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:16:56.914295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f6cd4a46-47fb-4089-973b-e6bc2abc062e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:56.922777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 12:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:16:57.477868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb83f0e-30ae-4f0b-bd6a-10be82c63b19 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:16:57.480476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1bb83f0e-30ae-4f0b-bd6a-10be82c63b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:16:57.496485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb83f0e-30ae-4f0b-bd6a-10be82c63b19 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:16:57.500497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dae8d1-44d4-4b95-ad43-b1ee53c57922 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:16:57.501178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dae8d1-44d4-4b95-ad43-b1ee53c57922 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:16:57.501453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dae8d1-44d4-4b95-ad43-b1ee53c57922 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:16:57.501720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dae8d1-44d4-4b95-ad43-b1ee53c57922 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:16:57.511967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dae8d1-44d4-4b95-ad43-b1ee53c57922 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:16:57.513526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dae8d1-44d4-4b95-ad43-b1ee53c57922 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:16:57.523942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 770/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:17:06.907594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=11 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:17:07.519395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:25.237098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b3e9-4455-4958-bba2-af101a9cd26a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.237796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b3e9-4455-4958-bba2-af101a9cd26a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.237796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b3e9-4455-4958-bba2-af101a9cd26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.265370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fb68b3e9-4455-4958-bba2-af101a9cd26a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.266861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fb68b3e9-4455-4958-bba2-af101a9cd26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.283058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 770/1540] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.284349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1227e5-6bf9-4f94-8ecc-428281d9c849 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.285553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1227e5-6bf9-4f94-8ecc-428281d9c849 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.286082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1227e5-6bf9-4f94-8ecc-428281d9c849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.311811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c2904-14f6-4a14-b324-bec75f121461 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.312174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c2904-14f6-4a14-b324-bec75f121461 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.312352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c2904-14f6-4a14-b324-bec75f121461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.319399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9a1227e5-6bf9-4f94-8ecc-428281d9c849 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.321368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9a1227e5-6bf9-4f94-8ecc-428281d9c849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.334807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 771/1541] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.338175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b25c2904-14f6-4a14-b324-bec75f121461 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.339413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b25c2904-14f6-4a14-b324-bec75f121461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.343400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722964ed-37cf-4cbf-b5dc-42a00bf92030 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.343998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722964ed-37cf-4cbf-b5dc-42a00bf92030 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.344492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722964ed-37cf-4cbf-b5dc-42a00bf92030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.353852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 771/1542] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.355125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf0655-43e8-4d96-99fa-61b61c27ec15 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.358511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf0655-43e8-4d96-99fa-61b61c27ec15 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.358843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf0655-43e8-4d96-99fa-61b61c27ec15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.383025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-722964ed-37cf-4cbf-b5dc-42a00bf92030 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.385439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-722964ed-37cf-4cbf-b5dc-42a00bf92030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.392805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6dcf0655-43e8-4d96-99fa-61b61c27ec15 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.393200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6dcf0655-43e8-4d96-99fa-61b61c27ec15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.411837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 772/1543] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.412690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf0655-43e8-4d96-99fa-61b61c27ec15 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.413359 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 772/1544] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.417589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722964ed-37cf-4cbf-b5dc-42a00bf92030 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.432021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf0655-43e8-4d96-99fa-61b61c27ec15 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:25.434183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722964ed-37cf-4cbf-b5dc-42a00bf92030 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:25.439502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2ad29-c38f-436a-9627-f715d69f3c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:25.440651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2ad29-c38f-436a-9627-f715d69f3c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.440794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2ad29-c38f-436a-9627-f715d69f3c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.440972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2ad29-c38f-436a-9627-f715d69f3c83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.441636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2ad29-c38f-436a-9627-f715d69f3c83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:25.441938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2ad29-c38f-436a-9627-f715d69f3c83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:25.443586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2ad29-c38f-436a-9627-f715d69f3c83 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:25.446884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:25.447792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.448283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.448769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.449744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:25.450270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:25.452130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:18:25.452130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Aug 31 12:18:25.452130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: Aug 31 12:18:25.453412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:25.478462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 773/1545] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:25.479779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b21e-dace-4d1b-a840-9a9dc49859a1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.480064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b21e-dace-4d1b-a840-9a9dc49859a1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.480470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b21e-dace-4d1b-a840-9a9dc49859a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.480754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 773/1546] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:18:25.489644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.508769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1d78b-304e-4e21-b5e5-03c0dc8e0e00 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:25.517121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2ab7b21e-dace-4d1b-a840-9a9dc49859a1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.518342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2ab7b21e-dace-4d1b-a840-9a9dc49859a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.519950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3737fb0-fc8e-477d-891c-2763f8fad1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:25.520935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3737fb0-fc8e-477d-891c-2763f8fad1ed admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.521428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3737fb0-fc8e-477d-891c-2763f8fad1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.521892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3737fb0-fc8e-477d-891c-2763f8fad1ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.523384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3737fb0-fc8e-477d-891c-2763f8fad1ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:25.525675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3737fb0-fc8e-477d-891c-2763f8fad1ed admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:25.535480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 774/1547] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.536890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7b21e-dace-4d1b-a840-9a9dc49859a1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.560283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7b21e-dace-4d1b-a840-9a9dc49859a1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:25.568374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:25.569018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.569296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.569565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.570477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:25.575282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:18:25.575282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Aug 31 12:18:25.575282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: Aug 31 12:18:25.575282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:25.624754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 774/1548] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:25.626503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c78a72-8999-4b62-b2f0-2e603cc30ad2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.626795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c78a72-8999-4b62-b2f0-2e603cc30ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.627071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c78a72-8999-4b62-b2f0-2e603cc30ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.632532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 775/1549] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:25.633887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.651751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c2c78a72-8999-4b62-b2f0-2e603cc30ad2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.653442 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c2c78a72-8999-4b62-b2f0-2e603cc30ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.715063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7195-2553-4372-a619-c81271935ff6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:25.719897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197be7a-e9ba-4e3a-9533-83b04e807105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:25.720571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197be7a-e9ba-4e3a-9533-83b04e807105 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.720851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197be7a-e9ba-4e3a-9533-83b04e807105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.721124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197be7a-e9ba-4e3a-9533-83b04e807105 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.721770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197be7a-e9ba-4e3a-9533-83b04e807105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:25.722081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197be7a-e9ba-4e3a-9533-83b04e807105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:25.723932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197be7a-e9ba-4e3a-9533-83b04e807105 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:25.724303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 775/1550] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.725804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a687224-14da-4c6c-b7da-cf45682983b4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.726080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a687224-14da-4c6c-b7da-cf45682983b4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.726346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a687224-14da-4c6c-b7da-cf45682983b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.744282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 776/1551] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:25.745814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5e44a-298e-4ef8-b008-5c2d9e46a05e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.746249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5e44a-298e-4ef8-b008-5c2d9e46a05e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.746670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5e44a-298e-4ef8-b008-5c2d9e46a05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.757152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0a687224-14da-4c6c-b7da-cf45682983b4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.758508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0a687224-14da-4c6c-b7da-cf45682983b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.770377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 776/1552] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.771731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a687224-14da-4c6c-b7da-cf45682983b4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.776763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7aa5e44a-298e-4ef8-b008-5c2d9e46a05e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.778969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7aa5e44a-298e-4ef8-b008-5c2d9e46a05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.786574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a687224-14da-4c6c-b7da-cf45682983b4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:25.791569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 777/1553] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.792943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5e44a-298e-4ef8-b008-5c2d9e46a05e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.793379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b6c16-65e0-4b12-a816-490165b8a4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:25.794282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b6c16-65e0-4b12-a816-490165b8a4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.794726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b6c16-65e0-4b12-a816-490165b8a4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.795173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b6c16-65e0-4b12-a816-490165b8a4a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.796257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b6c16-65e0-4b12-a816-490165b8a4a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:25.797985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-412b6c16-65e0-4b12-a816-490165b8a4a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:18:25.797985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Aug 31 12:18:25.797985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: Aug 31 12:18:25.800436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b6c16-65e0-4b12-a816-490165b8a4a7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:25.806579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5e44a-298e-4ef8-b008-5c2d9e46a05e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:25.811463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b4e01-231e-46dd-a445-2edf5404df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:25.812061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4e01-231e-46dd-a445-2edf5404df47 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.812332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4e01-231e-46dd-a445-2edf5404df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.812600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4e01-231e-46dd-a445-2edf5404df47 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.813249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b4e01-231e-46dd-a445-2edf5404df47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:25.815673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b4e01-231e-46dd-a445-2edf5404df47 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:25.829031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 777/1554] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:25.829031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856e1e-40c7-48ce-beb6-a9349187ca7f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.829031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856e1e-40c7-48ce-beb6-a9349187ca7f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.829031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856e1e-40c7-48ce-beb6-a9349187ca7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.844866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 778/1555] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/users => generated 294 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:25.844866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c14ab-7c63-448b-9b01-fd446707cb05 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.844866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c14ab-7c63-448b-9b01-fd446707cb05 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.844866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c14ab-7c63-448b-9b01-fd446707cb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.849926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a1856e1e-40c7-48ce-beb6-a9349187ca7f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.851181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a1856e1e-40c7-48ce-beb6-a9349187ca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.860030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 778/1556] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.861247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcaa0f-8dce-4e44-b40f-783dcb3fbda9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.861531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcaa0f-8dce-4e44-b40f-783dcb3fbda9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.861799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcaa0f-8dce-4e44-b40f-783dcb3fbda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.864868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9a7c14ab-7c63-448b-9b01-fd446707cb05 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.866140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9a7c14ab-7c63-448b-9b01-fd446707cb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.878814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 779/1557] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.879627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c14ab-7c63-448b-9b01-fd446707cb05 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.881915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d4fcaa0f-8dce-4e44-b40f-783dcb3fbda9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.883354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d4fcaa0f-8dce-4e44-b40f-783dcb3fbda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.889456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c14ab-7c63-448b-9b01-fd446707cb05 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:25.895347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 779/1558] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.895347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450330-4345-4721-8bd6-9c88c1cfa20e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.895347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450330-4345-4721-8bd6-9c88c1cfa20e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.895347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450330-4345-4721-8bd6-9c88c1cfa20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.898698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233c1b-ac13-46d8-976e-565432d98ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:25.899431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233c1b-ac13-46d8-976e-565432d98ded admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.899719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233c1b-ac13-46d8-976e-565432d98ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.899990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233c1b-ac13-46d8-976e-565432d98ded admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.900629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86233c1b-ac13-46d8-976e-565432d98ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:25.902544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86233c1b-ac13-46d8-976e-565432d98ded admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:25.917435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6b450330-4345-4721-8bd6-9c88c1cfa20e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.918661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6b450330-4345-4721-8bd6-9c88c1cfa20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.927489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 780/1559] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:25.931897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 780/1560] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:25.932343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910adef4-dea8-46d3-91b5-3322f2398e0c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:25.932682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910adef4-dea8-46d3-91b5-3322f2398e0c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:25.932960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910adef4-dea8-46d3-91b5-3322f2398e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:25.933082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233c1b-ac13-46d8-976e-565432d98ded admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:25.952537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-910adef4-dea8-46d3-91b5-3322f2398e0c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:25.953746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-910adef4-dea8-46d3-91b5-3322f2398e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:25.980532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:26.002615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233c1b-ac13-46d8-976e-565432d98ded admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.005605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:26.008323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b259519-7bd6-4765-9d0f-bd66b1c3b7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.008947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b259519-7bd6-4765-9d0f-bd66b1c3b7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.009237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b259519-7bd6-4765-9d0f-bd66b1c3b7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.009501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b259519-7bd6-4765-9d0f-bd66b1c3b7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.010155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b259519-7bd6-4765-9d0f-bd66b1c3b7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.011834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b259519-7bd6-4765-9d0f-bd66b1c3b7f3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.014912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 781/1561] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:26.015778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910adef4-dea8-46d3-91b5-3322f2398e0c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.023944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 781/1562] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:26.025201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b578b-ae25-4ed2-a1e2-5b96ae88d16d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.025486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b578b-ae25-4ed2-a1e2-5b96ae88d16d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.025757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b578b-ae25-4ed2-a1e2-5b96ae88d16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.025869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910adef4-dea8-46d3-91b5-3322f2398e0c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.030627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f9d9f-8ae4-4360-99ac-6e18d1b0b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.031251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f9d9f-8ae4-4360-99ac-6e18d1b0b938 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.031532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f9d9f-8ae4-4360-99ac-6e18d1b0b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.031804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f9d9f-8ae4-4360-99ac-6e18d1b0b938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.032444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f9d9f-8ae4-4360-99ac-6e18d1b0b938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.034093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f9d9f-8ae4-4360-99ac-6e18d1b0b938 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.044907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 782/1563] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:26.045725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f9d9f-8ae4-4360-99ac-6e18d1b0b938 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.047887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-218b578b-ae25-4ed2-a1e2-5b96ae88d16d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:26.049790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-218b578b-ae25-4ed2-a1e2-5b96ae88d16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:26.056397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f9d9f-8ae4-4360-99ac-6e18d1b0b938 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.061064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.061844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.061955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 782/1564] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:26.062061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.062168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.062814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:26.063119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.063515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e90d9-b1d3-4235-a879-c06bdf26e315 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.063925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e90d9-b1d3-4235-a879-c06bdf26e315 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.064322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e90d9-b1d3-4235-a879-c06bdf26e315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.064758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.086044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 783/1565] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:26.086044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.088486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-012e90d9-b1d3-4235-a879-c06bdf26e315 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:26.089672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-012e90d9-b1d3-4235-a879-c06bdf26e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:26.095257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb7dd-635e-459f-bc27-b87e95bc6f0f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.098057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 783/1566] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:26.099259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e90d9-b1d3-4235-a879-c06bdf26e315 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.103448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.104065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.104340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.104610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.105280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:26.105587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.107317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.110915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e90d9-b1d3-4235-a879-c06bdf26e315 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.115966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df52735-6094-431f-a383-983e11fcf6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.116571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df52735-6094-431f-a383-983e11fcf6cd admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.116853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df52735-6094-431f-a383-983e11fcf6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.117126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df52735-6094-431f-a383-983e11fcf6cd admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.117779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df52735-6094-431f-a383-983e11fcf6cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.119725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df52735-6094-431f-a383-983e11fcf6cd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.126010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 784/1567] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:26.126010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.132187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a467e-412e-46df-b72b-7b3e344033ef admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.137260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.137865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.138177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.138423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.139134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:26.139452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.141112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.147504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 784/1568] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.151508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df52735-6094-431f-a383-983e11fcf6cd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.155376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 785/1569] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:26.156109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.162131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df52735-6094-431f-a383-983e11fcf6cd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.167141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5140f-c854-4753-a2d3-507a660fa9c1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.168121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca962f-9a12-404e-928a-13ca59dddbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.168759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca962f-9a12-404e-928a-13ca59dddbdb admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.169038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca962f-9a12-404e-928a-13ca59dddbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.169305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca962f-9a12-404e-928a-13ca59dddbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.169984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca962f-9a12-404e-928a-13ca59dddbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.171613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca962f-9a12-404e-928a-13ca59dddbdb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.173051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093e4aa-7697-4ac4-8a23-a3649e4a7c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.173676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093e4aa-7697-4ac4-8a23-a3649e4a7c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.173960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093e4aa-7697-4ac4-8a23-a3649e4a7c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.174230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093e4aa-7697-4ac4-8a23-a3649e4a7c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.174860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093e4aa-7697-4ac4-8a23-a3649e4a7c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.176620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093e4aa-7697-4ac4-8a23-a3649e4a7c67 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.183444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 785/1570] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:26.184318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca962f-9a12-404e-928a-13ca59dddbdb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.187688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 786/1571] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:26.188526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093e4aa-7697-4ac4-8a23-a3649e4a7c67 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.196493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca962f-9a12-404e-928a-13ca59dddbdb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.201230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093e4aa-7697-4ac4-8a23-a3649e4a7c67 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.201720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.202326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.202604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.202883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.203543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:26.203846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.205452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.208901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334dad0-adb8-4dae-b9f9-245ba6b10231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.210095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334dad0-adb8-4dae-b9f9-245ba6b10231 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.210520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334dad0-adb8-4dae-b9f9-245ba6b10231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.210842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334dad0-adb8-4dae-b9f9-245ba6b10231 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.211690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334dad0-adb8-4dae-b9f9-245ba6b10231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.213833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334dad0-adb8-4dae-b9f9-245ba6b10231 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.219213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 786/1572] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:26.220174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.230798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb054cd-d60f-4c14-876d-a73b2f4dfdee admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.235532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.236139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.236410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.236800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.237312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:26.237893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.239862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.251174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 787/1573] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/projects => generated 355 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.251174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334dad0-adb8-4dae-b9f9-245ba6b10231 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.257660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 787/1574] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:26.258689 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.269068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739edc3-4e43-47b9-97c8-4c812a54968e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.318120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334dad0-adb8-4dae-b9f9-245ba6b10231 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.323773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a9a96-457f-4227-a65b-c3d1bf775ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.324405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a9a96-457f-4227-a65b-c3d1bf775ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.324665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a9a96-457f-4227-a65b-c3d1bf775ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.324936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a9a96-457f-4227-a65b-c3d1bf775ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.325642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a9a96-457f-4227-a65b-c3d1bf775ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.327885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a9a96-457f-4227-a65b-c3d1bf775ab2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.330327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fd4de-eef3-47f1-b1f7-d3842c189b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.330937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fd4de-eef3-47f1-b1f7-d3842c189b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.331215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fd4de-eef3-47f1-b1f7-d3842c189b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.331552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fd4de-eef3-47f1-b1f7-d3842c189b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.332792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fd4de-eef3-47f1-b1f7-d3842c189b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.334394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fd4de-eef3-47f1-b1f7-d3842c189b68 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.344876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 788/1575] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:26.345467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fd4de-eef3-47f1-b1f7-d3842c189b68 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.356137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fd4de-eef3-47f1-b1f7-d3842c189b68 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.362204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 788/1576] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.362204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a9a96-457f-4227-a65b-c3d1bf775ab2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.362403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f94423-0f92-4001-aa5e-828308320852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.362403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f94423-0f92-4001-aa5e-828308320852 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.362403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f94423-0f92-4001-aa5e-828308320852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.362403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f94423-0f92-4001-aa5e-828308320852 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.362974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f94423-0f92-4001-aa5e-828308320852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.365555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f94423-0f92-4001-aa5e-828308320852 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.396437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 789/1577] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.398097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f94423-0f92-4001-aa5e-828308320852 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.429168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a9a96-457f-4227-a65b-c3d1bf775ab2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.434342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:26.446795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:26.466406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f94423-0f92-4001-aa5e-828308320852 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.471043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.471752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.472031 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.472328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] PATH_INFO: `/v3/projects/f16c9f15d25f47a8a986290c237084d1/users/09b76f14426c4241aa49812f06f35cd1/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.487194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16c9f15d25f47a8a986290c237084d1, user_id=09b76f14426c4241aa49812f06f35cd1, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.488143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0924a2-935f-4f87-af21-5f31e390cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.488808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0924a2-935f-4f87-af21-5f31e390cd17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.488943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:18:26.488943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Aug 31 12:18:26.488943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: Aug 31 12:18:26.489229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0924a2-935f-4f87-af21-5f31e390cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.489341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0924a2-935f-4f87-af21-5f31e390cd17 admin admin] PATH_INFO: `/v3/projects/27224908eebd4b759efd8c6b24bb32a9/users/fdc5e43a88d04557b81c95ce3e22e5a7/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.491251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.504618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0924a2-935f-4f87-af21-5f31e390cd17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27224908eebd4b759efd8c6b24bb32a9, user_id=fdc5e43a88d04557b81c95ce3e22e5a7, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.507395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0924a2-935f-4f87-af21-5f31e390cd17 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.507512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 790/1578] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:26 2022] PUT /identity/v3/projects/f16c9f15d25f47a8a986290c237084d1/users/09b76f14426c4241aa49812f06f35cd1/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:26.508602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.523585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 789/1579] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:26 2022] PUT /identity/v3/projects/27224908eebd4b759efd8c6b24bb32a9/users/fdc5e43a88d04557b81c95ce3e22e5a7/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:26.524544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0924a2-935f-4f87-af21-5f31e390cd17 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.574077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693f8f-a887-4c20-9252-3f7923317964 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.598512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0924a2-935f-4f87-af21-5f31e390cd17 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.603173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430dd3ac-6881-4e70-9315-32e3c8b06261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.603810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430dd3ac-6881-4e70-9315-32e3c8b06261 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.604088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430dd3ac-6881-4e70-9315-32e3c8b06261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.604357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430dd3ac-6881-4e70-9315-32e3c8b06261 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.604987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430dd3ac-6881-4e70-9315-32e3c8b06261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.606900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430dd3ac-6881-4e70-9315-32e3c8b06261 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.628960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 790/1580] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/users => generated 278 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.629765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430dd3ac-6881-4e70-9315-32e3c8b06261 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.630687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cdcca-9776-416e-b2a0-69701dec8a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.631331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdcca-9776-416e-b2a0-69701dec8a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.631607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdcca-9776-416e-b2a0-69701dec8a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.631873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdcca-9776-416e-b2a0-69701dec8a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.632564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cdcca-9776-416e-b2a0-69701dec8a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.634572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cdcca-9776-416e-b2a0-69701dec8a62 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.641896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430dd3ac-6881-4e70-9315-32e3c8b06261 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.646998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503c709-2fb9-42cc-9773-20e477ff0087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.648321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503c709-2fb9-42cc-9773-20e477ff0087 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.648321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503c709-2fb9-42cc-9773-20e477ff0087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.648321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503c709-2fb9-42cc-9773-20e477ff0087 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.649005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503c709-2fb9-42cc-9773-20e477ff0087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.650900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503c709-2fb9-42cc-9773-20e477ff0087 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.664294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 791/1581] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/projects => generated 363 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.665202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cdcca-9776-416e-b2a0-69701dec8a62 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.687458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 791/1582] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.688406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503c709-2fb9-42cc-9773-20e477ff0087 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.729407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:26.733579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cdcca-9776-416e-b2a0-69701dec8a62 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.749881 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503c709-2fb9-42cc-9773-20e477ff0087 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.754790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf7c37-6fd0-45c9-b9f4-9df8b6934f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.755436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7c37-6fd0-45c9-b9f4-9df8b6934f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.755815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7c37-6fd0-45c9-b9f4-9df8b6934f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.755991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7c37-6fd0-45c9-b9f4-9df8b6934f7a admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.756619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf7c37-6fd0-45c9-b9f4-9df8b6934f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.758577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf7c37-6fd0-45c9-b9f4-9df8b6934f7a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.766725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:26.785133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 792/1583] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.785133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf7c37-6fd0-45c9-b9f4-9df8b6934f7a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.792514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf7c37-6fd0-45c9-b9f4-9df8b6934f7a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.794592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf7958-093b-41da-ba95-087f4cc17919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.795256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf7958-093b-41da-ba95-087f4cc17919 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.795539 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf7958-093b-41da-ba95-087f4cc17919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.795805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf7958-093b-41da-ba95-087f4cc17919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.796437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf7958-093b-41da-ba95-087f4cc17919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.797356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df7a25-327c-4f64-ae43-11e5c8ec7f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.797951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df7a25-327c-4f64-ae43-11e5c8ec7f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.798062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf7958-093b-41da-ba95-087f4cc17919 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.798232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df7a25-327c-4f64-ae43-11e5c8ec7f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.798505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df7a25-327c-4f64-ae43-11e5c8ec7f50 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.799130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df7a25-327c-4f64-ae43-11e5c8ec7f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.801138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df7a25-327c-4f64-ae43-11e5c8ec7f50 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.808881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 792/1584] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:26.810278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508f6c-2d06-4fc4-aa13-6b350ead70f6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.810999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508f6c-2d06-4fc4-aa13-6b350ead70f6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.811743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508f6c-2d06-4fc4-aa13-6b350ead70f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.827058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 793/1585] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/users => generated 296 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.827058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199681f4-4e64-4358-800c-96e617bb0355 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.827058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199681f4-4e64-4358-800c-96e617bb0355 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.827058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199681f4-4e64-4358-800c-96e617bb0355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.835566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-70508f6c-2d06-4fc4-aa13-6b350ead70f6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:26.836765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-70508f6c-2d06-4fc4-aa13-6b350ead70f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:26.844595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 793/1586] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:26.845484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70508f6c-2d06-4fc4-aa13-6b350ead70f6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.856273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70508f6c-2d06-4fc4-aa13-6b350ead70f6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.860072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-199681f4-4e64-4358-800c-96e617bb0355 None None] MFA Rules not processed for user `fdc5e43a88d04557b81c95ce3e22e5a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:26.860939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973fbbd-bfd3-449b-86c8-86322bb971b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.861266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-199681f4-4e64-4358-800c-96e617bb0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:26.861540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973fbbd-bfd3-449b-86c8-86322bb971b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.861820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973fbbd-bfd3-449b-86c8-86322bb971b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.862115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973fbbd-bfd3-449b-86c8-86322bb971b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.862743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973fbbd-bfd3-449b-86c8-86322bb971b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.864409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973fbbd-bfd3-449b-86c8-86322bb971b7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.878988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 794/1587] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:26.878988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973fbbd-bfd3-449b-86c8-86322bb971b7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.889324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973fbbd-bfd3-449b-86c8-86322bb971b7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.894380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836e9ec-8ac2-4efb-983a-586975a6c4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.895206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836e9ec-8ac2-4efb-983a-586975a6c4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.895577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836e9ec-8ac2-4efb-983a-586975a6c4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.895910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836e9ec-8ac2-4efb-983a-586975a6c4e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.896608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836e9ec-8ac2-4efb-983a-586975a6c4e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.898499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836e9ec-8ac2-4efb-983a-586975a6c4e3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.925225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 795/1588] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/users => generated 282 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.925225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836e9ec-8ac2-4efb-983a-586975a6c4e3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.931628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 794/1589] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:26.936457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199681f4-4e64-4358-800c-96e617bb0355 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.936573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836e9ec-8ac2-4efb-983a-586975a6c4e3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.942721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692e53-0aca-4f5d-91a9-e6328a088beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.943388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692e53-0aca-4f5d-91a9-e6328a088beb admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.943702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692e53-0aca-4f5d-91a9-e6328a088beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.943975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692e53-0aca-4f5d-91a9-e6328a088beb admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.944609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc692e53-0aca-4f5d-91a9-e6328a088beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.945091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199681f4-4e64-4358-800c-96e617bb0355 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.946618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc692e53-0aca-4f5d-91a9-e6328a088beb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.949787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0019b0-3680-479a-8a1b-20f719e1016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.950398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0019b0-3680-479a-8a1b-20f719e1016c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.950674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0019b0-3680-479a-8a1b-20f719e1016c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.950943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0019b0-3680-479a-8a1b-20f719e1016c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.951631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0019b0-3680-479a-8a1b-20f719e1016c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.953727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0019b0-3680-479a-8a1b-20f719e1016c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:26.966202 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 795/1590] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:26.967051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0019b0-3680-479a-8a1b-20f719e1016c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.978334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 796/1591] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:26 2022] POST /identity/v3/users => generated 289 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:26.978560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0019b0-3680-479a-8a1b-20f719e1016c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.979299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692e53-0aca-4f5d-91a9-e6328a088beb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:26.985419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c156b25-2711-4f3f-8857-330b58d3e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.986139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156b25-2711-4f3f-8857-330b58d3e983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.986419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156b25-2711-4f3f-8857-330b58d3e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.986696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156b25-2711-4f3f-8857-330b58d3e983 admin admin] PATH_INFO: `/v3/projects/16f3955065954f8f860bada6261223c4/users/4c3e5c1776b8470a9b677bac8dfbc9e6/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.989795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692e53-0aca-4f5d-91a9-e6328a088beb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:26.994424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:26.995017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:26.995338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:26.995609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:26.996274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:26.997911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.002075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c156b25-2711-4f3f-8857-330b58d3e983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f3955065954f8f860bada6261223c4, user_id=4c3e5c1776b8470a9b677bac8dfbc9e6, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.004929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c156b25-2711-4f3f-8857-330b58d3e983 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.011166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 797/1592] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:27.011166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.011166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:27.022617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 796/1593] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:26 2022] PUT /identity/v3/projects/16f3955065954f8f860bada6261223c4/users/4c3e5c1776b8470a9b677bac8dfbc9e6/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:27.023396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c156b25-2711-4f3f-8857-330b58d3e983 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.068382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add60d-bc30-419f-bdc2-e7e92be86e3d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.084481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c156b25-2711-4f3f-8857-330b58d3e983 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.089317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8516-ef20-41c6-8bb0-dcea5fd83328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.089932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8516-ef20-41c6-8bb0-dcea5fd83328 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.090213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8516-ef20-41c6-8bb0-dcea5fd83328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.090487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8516-ef20-41c6-8bb0-dcea5fd83328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.091132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a8516-ef20-41c6-8bb0-dcea5fd83328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.093131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a8516-ef20-41c6-8bb0-dcea5fd83328 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.103672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 797/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:27.104492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8516-ef20-41c6-8bb0-dcea5fd83328 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.113857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8516-ef20-41c6-8bb0-dcea5fd83328 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.118341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef5101-4770-41d2-8133-b75ab9259b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.118937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef5101-4770-41d2-8133-b75ab9259b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.119211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef5101-4770-41d2-8133-b75ab9259b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.119365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c43649-407d-4064-88ec-8098f4a4e841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.119490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef5101-4770-41d2-8133-b75ab9259b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.119864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c43649-407d-4064-88ec-8098f4a4e841 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.120125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef5101-4770-41d2-8133-b75ab9259b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.120246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c43649-407d-4064-88ec-8098f4a4e841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.120387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c43649-407d-4064-88ec-8098f4a4e841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.121719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef5101-4770-41d2-8133-b75ab9259b6d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.128261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:27.131136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c43649-407d-4064-88ec-8098f4a4e841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.135843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 798/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:27.136003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c43649-407d-4064-88ec-8098f4a4e841 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.136187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4aa77-a84a-4690-a19d-27b8bcf3fccf None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.136187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4aa77-a84a-4690-a19d-27b8bcf3fccf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.136187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4aa77-a84a-4690-a19d-27b8bcf3fccf None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.136187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 799/1596] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 31 12:18:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:18:27.136472 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4aa77-a84a-4690-a19d-27b8bcf3fccf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.148694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 798/1597] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:27.149248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4aa77-a84a-4690-a19d-27b8bcf3fccf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.149511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c43649-407d-4064-88ec-8098f4a4e841 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.153940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa8bfb-86b6-43cb-b8b5-da4b3da10b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.154599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa8bfb-86b6-43cb-b8b5-da4b3da10b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.154774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa8bfb-86b6-43cb-b8b5-da4b3da10b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.155043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa8bfb-86b6-43cb-b8b5-da4b3da10b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.155853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa8bfb-86b6-43cb-b8b5-da4b3da10b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.157279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa8bfb-86b6-43cb-b8b5-da4b3da10b0d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.159276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c43649-407d-4064-88ec-8098f4a4e841 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.163742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:27.164999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5b0e2-108b-41f2-b93b-235d00d894c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.165640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5b0e2-108b-41f2-b93b-235d00d894c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.165950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5b0e2-108b-41f2-b93b-235d00d894c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.166187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5b0e2-108b-41f2-b93b-235d00d894c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.167199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5b0e2-108b-41f2-b93b-235d00d894c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.168836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5b0e2-108b-41f2-b93b-235d00d894c8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.173114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 800/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:27.174027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa8bfb-86b6-43cb-b8b5-da4b3da10b0d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.181399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 799/1599] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:27.182550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b73a9-d364-449c-9e5b-59353e3deed2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.182837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b73a9-d364-449c-9e5b-59353e3deed2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.183354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b73a9-d364-449c-9e5b-59353e3deed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.188897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa8bfb-86b6-43cb-b8b5-da4b3da10b0d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.194145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c274f0-294d-445d-a52d-0ed40e77789b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.194893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c274f0-294d-445d-a52d-0ed40e77789b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.195169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c274f0-294d-445d-a52d-0ed40e77789b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.195600 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c274f0-294d-445d-a52d-0ed40e77789b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.196116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c274f0-294d-445d-a52d-0ed40e77789b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.197709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c274f0-294d-445d-a52d-0ed40e77789b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.203433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7c8b73a9-d364-449c-9e5b-59353e3deed2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:27.204639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7c8b73a9-d364-449c-9e5b-59353e3deed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:27.208743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 801/1600] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:27.212215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c274f0-294d-445d-a52d-0ed40e77789b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.223530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 800/1601] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:27.224540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b73a9-d364-449c-9e5b-59353e3deed2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.231192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c274f0-294d-445d-a52d-0ed40e77789b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.236086 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b73a9-d364-449c-9e5b-59353e3deed2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.237208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c92cc1-6fb1-40ef-b43d-533f2ee4107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.238660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c92cc1-6fb1-40ef-b43d-533f2ee4107c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.239022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c92cc1-6fb1-40ef-b43d-533f2ee4107c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.239348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c92cc1-6fb1-40ef-b43d-533f2ee4107c admin admin] PATH_INFO: `/v3/projects/880fc0b619f64346903ff359716ba22a/users/7f4b08ec590749dcb8618b09a4c6c969/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.241655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43df94-f61e-4e93-a1ae-b0ecb8f0f471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.242554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43df94-f61e-4e93-a1ae-b0ecb8f0f471 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.242554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43df94-f61e-4e93-a1ae-b0ecb8f0f471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.242786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43df94-f61e-4e93-a1ae-b0ecb8f0f471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.243668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43df94-f61e-4e93-a1ae-b0ecb8f0f471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.245588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43df94-f61e-4e93-a1ae-b0ecb8f0f471 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.259271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 801/1602] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:27.259271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2b5e9-2664-4f22-b43e-8530f640f935 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.259271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2b5e9-2664-4f22-b43e-8530f640f935 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.259271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2b5e9-2664-4f22-b43e-8530f640f935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.261558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c92cc1-6fb1-40ef-b43d-533f2ee4107c admin admin] RBAC: Authorizing `identity:create_grant(project_id=880fc0b619f64346903ff359716ba22a, user_id=7f4b08ec590749dcb8618b09a4c6c969, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.264473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c92cc1-6fb1-40ef-b43d-533f2ee4107c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.281979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 802/1603] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:27 2022] PUT /identity/v3/projects/880fc0b619f64346903ff359716ba22a/users/7f4b08ec590749dcb8618b09a4c6c969/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:27.284498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c92cc1-6fb1-40ef-b43d-533f2ee4107c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.306575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-10a2b5e9-2664-4f22-b43e-8530f640f935 None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:27.307833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-10a2b5e9-2664-4f22-b43e-8530f640f935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:27.377193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c92cc1-6fb1-40ef-b43d-533f2ee4107c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.382538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099d6e8-996d-4bc5-af7e-f8f77fd81851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.383345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099d6e8-996d-4bc5-af7e-f8f77fd81851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.383647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099d6e8-996d-4bc5-af7e-f8f77fd81851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.383922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099d6e8-996d-4bc5-af7e-f8f77fd81851 admin admin] PATH_INFO: `/v3/projects/c4730ec62c464218b57cdfdd5ee8a36f/users/34cfab2f4eca48e880b8277897a0ce7b/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.407386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099d6e8-996d-4bc5-af7e-f8f77fd81851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4730ec62c464218b57cdfdd5ee8a36f, user_id=34cfab2f4eca48e880b8277897a0ce7b, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.410426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099d6e8-996d-4bc5-af7e-f8f77fd81851 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.426304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 803/1604] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:27 2022] PUT /identity/v3/projects/c4730ec62c464218b57cdfdd5ee8a36f/users/34cfab2f4eca48e880b8277897a0ce7b/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:27.427584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099d6e8-996d-4bc5-af7e-f8f77fd81851 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.432268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 802/1605] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 31 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:27.433935 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2b5e9-2664-4f22-b43e-8530f640f935 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.454308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2b5e9-2664-4f22-b43e-8530f640f935 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.493486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099d6e8-996d-4bc5-af7e-f8f77fd81851 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.498495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816ed5e-6740-413c-b582-7f930a9b2584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.499107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816ed5e-6740-413c-b582-7f930a9b2584 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.499421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816ed5e-6740-413c-b582-7f930a9b2584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.499692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816ed5e-6740-413c-b582-7f930a9b2584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.500362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816ed5e-6740-413c-b582-7f930a9b2584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.501941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816ed5e-6740-413c-b582-7f930a9b2584 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.515206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 804/1606] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:27.516156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816ed5e-6740-413c-b582-7f930a9b2584 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.540042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce512dff-3268-4f97-be50-d7a8b2f81d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.540823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce512dff-3268-4f97-be50-d7a8b2f81d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.541208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce512dff-3268-4f97-be50-d7a8b2f81d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.541556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce512dff-3268-4f97-be50-d7a8b2f81d37 admin admin] PATH_INFO: `/v3/projects/8687bcf9d2a242b5af329bce4a8ad6d9/users/a8320be5e9644b6699ac526913e109eb/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.560005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce512dff-3268-4f97-be50-d7a8b2f81d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8687bcf9d2a242b5af329bce4a8ad6d9, user_id=a8320be5e9644b6699ac526913e109eb, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.562874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce512dff-3268-4f97-be50-d7a8b2f81d37 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.571026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:27.578976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 803/1607] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:27 2022] PUT /identity/v3/projects/8687bcf9d2a242b5af329bce4a8ad6d9/users/a8320be5e9644b6699ac526913e109eb/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:27.580092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce512dff-3268-4f97-be50-d7a8b2f81d37 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.581678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816ed5e-6740-413c-b582-7f930a9b2584 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.616114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:27.654706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195927d5-8653-43ee-aeb1-3956221f0d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.655503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195927d5-8653-43ee-aeb1-3956221f0d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.655913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195927d5-8653-43ee-aeb1-3956221f0d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.656252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195927d5-8653-43ee-aeb1-3956221f0d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.666340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce512dff-3268-4f97-be50-d7a8b2f81d37 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.671505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195927d5-8653-43ee-aeb1-3956221f0d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.673046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195927d5-8653-43ee-aeb1-3956221f0d30 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.674167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0cdbe-868c-4983-afe5-f1b3ea277cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.675029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0cdbe-868c-4983-afe5-f1b3ea277cce admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.675550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0cdbe-868c-4983-afe5-f1b3ea277cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.676254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0cdbe-868c-4983-afe5-f1b3ea277cce admin admin] PATH_INFO: `/v3/projects/1d7219a4c67b439ba225a0397c3c2505/users/d22acbf59be246d487e31ba6400bb5ba/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.687890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 805/1608] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:27.689515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a168-ae91-4834-8f34-80364e89fe83 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.689877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a168-ae91-4834-8f34-80364e89fe83 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.690264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a168-ae91-4834-8f34-80364e89fe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.697097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0cdbe-868c-4983-afe5-f1b3ea277cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7219a4c67b439ba225a0397c3c2505, user_id=d22acbf59be246d487e31ba6400bb5ba, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.700189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0cdbe-868c-4983-afe5-f1b3ea277cce admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.718556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 804/1609] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:27 2022] PUT /identity/v3/projects/1d7219a4c67b439ba225a0397c3c2505/users/d22acbf59be246d487e31ba6400bb5ba/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:27.719900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0cdbe-868c-4983-afe5-f1b3ea277cce admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.746877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b3f7a168-ae91-4834-8f34-80364e89fe83 None None] MFA Rules not processed for user `7f4b08ec590749dcb8618b09a4c6c969`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:27.748247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b3f7a168-ae91-4834-8f34-80364e89fe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:27.821715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0cdbe-868c-4983-afe5-f1b3ea277cce admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:27.833627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 806/1610] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:27.834896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a6f7-c878-4e69-9fd5-1c616065b99a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.835092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a6f7-c878-4e69-9fd5-1c616065b99a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.835337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a6f7-c878-4e69-9fd5-1c616065b99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.837862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f783d-9145-4a9e-b442-7cd882b109a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:27.838987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f783d-9145-4a9e-b442-7cd882b109a4 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.839313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f783d-9145-4a9e-b442-7cd882b109a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.839590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f783d-9145-4a9e-b442-7cd882b109a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.867609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9de1a6f7-c878-4e69-9fd5-1c616065b99a None None] MFA Rules not processed for user `34cfab2f4eca48e880b8277897a0ce7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:27.869376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9de1a6f7-c878-4e69-9fd5-1c616065b99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:27.905846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f783d-9145-4a9e-b442-7cd882b109a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:27.907867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f783d-9145-4a9e-b442-7cd882b109a4 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:27.918618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 805/1611] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:27.919356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f783d-9145-4a9e-b442-7cd882b109a4 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:27.970562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 807/1612] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:27.973851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db149f8-317d-4330-9757-56bc1a65b9b2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:27.974200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db149f8-317d-4330-9757-56bc1a65b9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:27.974473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db149f8-317d-4330-9757-56bc1a65b9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:27.978720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:27.999191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5db149f8-317d-4330-9757-56bc1a65b9b2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:28.001042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5db149f8-317d-4330-9757-56bc1a65b9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:28.008440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f783d-9145-4a9e-b442-7cd882b109a4 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.011381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 808/1613] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:28.013004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c625e-b067-4442-af17-19da50a315cd None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.013485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c625e-b067-4442-af17-19da50a315cd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.014573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c625e-b067-4442-af17-19da50a315cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.021798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a6a3d-bd10-41bb-89b5-97e557bf0260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.022482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a6a3d-bd10-41bb-89b5-97e557bf0260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.022762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a6a3d-bd10-41bb-89b5-97e557bf0260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.023085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a6a3d-bd10-41bb-89b5-97e557bf0260 admin admin] PATH_INFO: `/v3/projects/15cb4f1fac4f4d728094c3c152c1382d/users/cace97d1adb44009acf2b7b093d1e885/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.044856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a6a3d-bd10-41bb-89b5-97e557bf0260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15cb4f1fac4f4d728094c3c152c1382d, user_id=cace97d1adb44009acf2b7b093d1e885, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.048928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a6a3d-bd10-41bb-89b5-97e557bf0260 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.056705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-260c625e-b067-4442-af17-19da50a315cd None None] MFA Rules not processed for user `d22acbf59be246d487e31ba6400bb5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:28.058448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-260c625e-b067-4442-af17-19da50a315cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:28.064764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:28.071294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 806/1614] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:27 2022] PUT /identity/v3/projects/15cb4f1fac4f4d728094c3c152c1382d/users/cace97d1adb44009acf2b7b093d1e885/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:28.072568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a6a3d-bd10-41bb-89b5-97e557bf0260 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.142895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 809/1615] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:28.143984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c625e-b067-4442-af17-19da50a315cd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.208759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a6a3d-bd10-41bb-89b5-97e557bf0260 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.213887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23920cad-9288-4df8-9520-4f39bd815ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.214512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23920cad-9288-4df8-9520-4f39bd815ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.214786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23920cad-9288-4df8-9520-4f39bd815ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.215049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23920cad-9288-4df8-9520-4f39bd815ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.216218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23920cad-9288-4df8-9520-4f39bd815ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.218323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23920cad-9288-4df8-9520-4f39bd815ba5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.246780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c625e-b067-4442-af17-19da50a315cd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.254121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc3ad8-8a4e-497d-8544-684686030ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.254815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3ad8-8a4e-497d-8544-684686030ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.255091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3ad8-8a4e-497d-8544-684686030ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.255486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3ad8-8a4e-497d-8544-684686030ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.256431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc3ad8-8a4e-497d-8544-684686030ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.258722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc3ad8-8a4e-497d-8544-684686030ee9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.264059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 807/1616] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/projects => generated 355 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:28.265182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23920cad-9288-4df8-9520-4f39bd815ba5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.335407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23920cad-9288-4df8-9520-4f39bd815ba5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.338710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 810/1617] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/projects => generated 363 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:28.341216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814a55-2c23-41c8-b70b-5d3b0be6edc3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.341788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814a55-2c23-41c8-b70b-5d3b0be6edc3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.342235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814a55-2c23-41c8-b70b-5d3b0be6edc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.343802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542dc12-0c59-4127-aa47-ca0391f7ccda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.344920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542dc12-0c59-4127-aa47-ca0391f7ccda service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.345369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542dc12-0c59-4127-aa47-ca0391f7ccda service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.345792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542dc12-0c59-4127-aa47-ca0391f7ccda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.371687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4d814a55-2c23-41c8-b70b-5d3b0be6edc3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:28.373821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4d814a55-2c23-41c8-b70b-5d3b0be6edc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:28.437433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542dc12-0c59-4127-aa47-ca0391f7ccda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.440155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542dc12-0c59-4127-aa47-ca0391f7ccda service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.453028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 808/1618] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:28.453926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542dc12-0c59-4127-aa47-ca0391f7ccda service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.460596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 811/1619] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:28.463019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644560b-01d5-44b2-9752-ff15a9815b6a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.463341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644560b-01d5-44b2-9752-ff15a9815b6a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.463617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644560b-01d5-44b2-9752-ff15a9815b6a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.465478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 812/1620] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 31 12:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:18:28.466265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644560b-01d5-44b2-9752-ff15a9815b6a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.484840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542dc12-0c59-4127-aa47-ca0391f7ccda service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.485485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644560b-01d5-44b2-9752-ff15a9815b6a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.495622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a44b0-79ad-46cd-a470-aefb2c2237ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.496343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a44b0-79ad-46cd-a470-aefb2c2237ea admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.496526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a44b0-79ad-46cd-a470-aefb2c2237ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.496792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a44b0-79ad-46cd-a470-aefb2c2237ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.497441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a44b0-79ad-46cd-a470-aefb2c2237ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.500637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b2e7e-a777-410b-b65d-ff36704feb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.500637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2e7e-a777-410b-b65d-ff36704feb3e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.501080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2e7e-a777-410b-b65d-ff36704feb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.501351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2e7e-a777-410b-b65d-ff36704feb3e admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.501985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b2e7e-a777-410b-b65d-ff36704feb3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.502185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a44b0-79ad-46cd-a470-aefb2c2237ea admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.504132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b2e7e-a777-410b-b65d-ff36704feb3e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.514127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:28.548629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 813/1621] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/users => generated 284 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:28.549294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b2e7e-a777-410b-b65d-ff36704feb3e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.560591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 809/1622] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:28.561510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a44b0-79ad-46cd-a470-aefb2c2237ea admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.570989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b2e7e-a777-410b-b65d-ff36704feb3e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.590990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:28.658783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a44b0-79ad-46cd-a470-aefb2c2237ea admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.668544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694c85a-e48b-4a87-ad1c-47a70fc85e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.669225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694c85a-e48b-4a87-ad1c-47a70fc85e3f service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.669488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694c85a-e48b-4a87-ad1c-47a70fc85e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.669759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694c85a-e48b-4a87-ad1c-47a70fc85e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.687662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9f582-4b4b-4c82-a9bc-af8eafb59db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.688372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f582-4b4b-4c82-a9bc-af8eafb59db1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.688654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f582-4b4b-4c82-a9bc-af8eafb59db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.689068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f582-4b4b-4c82-a9bc-af8eafb59db1 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.689851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9f582-4b4b-4c82-a9bc-af8eafb59db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.692427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9f582-4b4b-4c82-a9bc-af8eafb59db1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.723451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 814/1623] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:28.723451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78eee1-76fe-41d6-879f-29c854f1ebf4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.723451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78eee1-76fe-41d6-879f-29c854f1ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.723451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78eee1-76fe-41d6-879f-29c854f1ebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.745669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fb78eee1-76fe-41d6-879f-29c854f1ebf4 None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:28.746821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fb78eee1-76fe-41d6-879f-29c854f1ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:28.767278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694c85a-e48b-4a87-ad1c-47a70fc85e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.769182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694c85a-e48b-4a87-ad1c-47a70fc85e3f service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.784695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 810/1624] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:28.785728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694c85a-e48b-4a87-ad1c-47a70fc85e3f service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.807930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694c85a-e48b-4a87-ad1c-47a70fc85e3f service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.819622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048a11-8b3e-4f45-829d-e73641b5e0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.820248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048a11-8b3e-4f45-829d-e73641b5e0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.820519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048a11-8b3e-4f45-829d-e73641b5e0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.820788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048a11-8b3e-4f45-829d-e73641b5e0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.822061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9048a11-8b3e-4f45-829d-e73641b5e0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.825785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9048a11-8b3e-4f45-829d-e73641b5e0b3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.864616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 811/1625] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:28.866466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048a11-8b3e-4f45-829d-e73641b5e0b3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.892995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 815/1626] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:28.898887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78eee1-76fe-41d6-879f-29c854f1ebf4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.900710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048a11-8b3e-4f45-829d-e73641b5e0b3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.920805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78eee1-76fe-41d6-879f-29c854f1ebf4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.921954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4b4f7-4b95-463e-8de4-746bba1f91bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.923785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b4f7-4b95-463e-8de4-746bba1f91bc admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.924153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b4f7-4b95-463e-8de4-746bba1f91bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.924771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b4f7-4b95-463e-8de4-746bba1f91bc admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.927480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4b4f7-4b95-463e-8de4-746bba1f91bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.931758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4b4f7-4b95-463e-8de4-746bba1f91bc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.939429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e920a-9856-40d3-9d6e-97cd903ee13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.940932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e920a-9856-40d3-9d6e-97cd903ee13d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.941374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e920a-9856-40d3-9d6e-97cd903ee13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.941819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e920a-9856-40d3-9d6e-97cd903ee13d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:28.943301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e920a-9856-40d3-9d6e-97cd903ee13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:28.945515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e920a-9856-40d3-9d6e-97cd903ee13d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:28.961706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 816/1627] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:28.962763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e920a-9856-40d3-9d6e-97cd903ee13d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:28.979450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e920a-9856-40d3-9d6e-97cd903ee13d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:28.986719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb7e67-5f80-4aeb-a5d1-a25d23bbcfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:28.988154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7e67-5f80-4aeb-a5d1-a25d23bbcfdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:28.988435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7e67-5f80-4aeb-a5d1-a25d23bbcfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:28.988705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7e67-5f80-4aeb-a5d1-a25d23bbcfdd admin admin] PATH_INFO: `/v3/projects/8230e51deebf4ca8bd11ec8ef5e860b3/users/cc2e08815035468f978939f8494711e5/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.005976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 812/1628] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:28 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:29.005976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4b4f7-4b95-463e-8de4-746bba1f91bc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:29.013392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb7e67-5f80-4aeb-a5d1-a25d23bbcfdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8230e51deebf4ca8bd11ec8ef5e860b3, user_id=cc2e08815035468f978939f8494711e5, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:29.017566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb7e67-5f80-4aeb-a5d1-a25d23bbcfdd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:29.044742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 817/1629] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:28 2022] PUT /identity/v3/projects/8230e51deebf4ca8bd11ec8ef5e860b3/users/cc2e08815035468f978939f8494711e5/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:29.044742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb7e67-5f80-4aeb-a5d1-a25d23bbcfdd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:29.143255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4b4f7-4b95-463e-8de4-746bba1f91bc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:29.168248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb7e67-5f80-4aeb-a5d1-a25d23bbcfdd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:29.172349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:29.180934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1b135-24d9-4130-86a3-796b31c0426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:29.182554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1b135-24d9-4130-86a3-796b31c0426d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:29.184102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1b135-24d9-4130-86a3-796b31c0426d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:29.184847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1b135-24d9-4130-86a3-796b31c0426d admin admin] PATH_INFO: `/v3/projects/82023d5e0854497b85b106a8dc30c1b8/users/73a4f8d757384d1381604f1158992a23/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.207892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:29.237316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b271064-2b4e-4600-a0b5-c65f6a51fad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:29.239030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271064-2b4e-4600-a0b5-c65f6a51fad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:29.239442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271064-2b4e-4600-a0b5-c65f6a51fad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:29.239636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271064-2b4e-4600-a0b5-c65f6a51fad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.258006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1b135-24d9-4130-86a3-796b31c0426d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82023d5e0854497b85b106a8dc30c1b8, user_id=73a4f8d757384d1381604f1158992a23, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:29.263534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1b135-24d9-4130-86a3-796b31c0426d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:29.301536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 818/1630] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:29 2022] PUT /identity/v3/projects/82023d5e0854497b85b106a8dc30c1b8/users/73a4f8d757384d1381604f1158992a23/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:29.303349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1b135-24d9-4130-86a3-796b31c0426d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:29.342181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b271064-2b4e-4600-a0b5-c65f6a51fad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:29.344004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b271064-2b4e-4600-a0b5-c65f6a51fad8 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:29.359884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 813/1631] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:29.360772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b271064-2b4e-4600-a0b5-c65f6a51fad8 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:29.372548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b271064-2b4e-4600-a0b5-c65f6a51fad8 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:29.445850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a779b32-e387-4258-bc18-5190186104f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:29.446899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a779b32-e387-4258-bc18-5190186104f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:29.447264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a779b32-e387-4258-bc18-5190186104f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:29.447608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a779b32-e387-4258-bc18-5190186104f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.451076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a779b32-e387-4258-bc18-5190186104f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:29.453976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a779b32-e387-4258-bc18-5190186104f8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:29.464975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1b135-24d9-4130-86a3-796b31c0426d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:29.476613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e09cd4-6991-41af-93d5-8a96a210468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:29.477642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e09cd4-6991-41af-93d5-8a96a210468e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:29.478118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e09cd4-6991-41af-93d5-8a96a210468e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:29.478566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e09cd4-6991-41af-93d5-8a96a210468e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.479650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e09cd4-6991-41af-93d5-8a96a210468e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:29.482992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e09cd4-6991-41af-93d5-8a96a210468e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:29.484323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 814/1632] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:29.485939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a779b32-e387-4258-bc18-5190186104f8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:29.510913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 819/1633] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:29.522087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e09cd4-6991-41af-93d5-8a96a210468e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:29.528718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a779b32-e387-4258-bc18-5190186104f8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:29.544471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e483141-b0df-45a4-b3d7-51993bfdcf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:29.548532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483141-b0df-45a4-b3d7-51993bfdcf02 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:29.549029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483141-b0df-45a4-b3d7-51993bfdcf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:29.549359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483141-b0df-45a4-b3d7-51993bfdcf02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.551616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e483141-b0df-45a4-b3d7-51993bfdcf02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:29.554572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e09cd4-6991-41af-93d5-8a96a210468e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:29.556422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e483141-b0df-45a4-b3d7-51993bfdcf02 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:29.580676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba8d88-84ee-472d-b3ff-97ecf7a2c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:29.582071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba8d88-84ee-472d-b3ff-97ecf7a2c10e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:29.583756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba8d88-84ee-472d-b3ff-97ecf7a2c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:29.585045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba8d88-84ee-472d-b3ff-97ecf7a2c10e admin admin] PATH_INFO: `/v3/projects/8230e51deebf4ca8bd11ec8ef5e860b3/users/cc2e08815035468f978939f8494711e5/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.588829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba8d88-84ee-472d-b3ff-97ecf7a2c10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8230e51deebf4ca8bd11ec8ef5e860b3, user_id=cc2e08815035468f978939f8494711e5, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:29.589602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 815/1634] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:29.591785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e483141-b0df-45a4-b3d7-51993bfdcf02 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:29.592886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba8d88-84ee-472d-b3ff-97ecf7a2c10e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:29.617489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e483141-b0df-45a4-b3d7-51993bfdcf02 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:29.631121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f76c46-ea2c-4953-a5e3-5406fc02f9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:29.634375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76c46-ea2c-4953-a5e3-5406fc02f9da admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:29.635177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76c46-ea2c-4953-a5e3-5406fc02f9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:29.635713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76c46-ea2c-4953-a5e3-5406fc02f9da admin admin] PATH_INFO: `/v3/projects/1e66e4e650d342bd865bc2953a2e2549/users/980dfdeca6df4b32a0784a9bff691aa3/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.645223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 820/1635] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:29 2022] PUT /identity/v3/projects/8230e51deebf4ca8bd11ec8ef5e860b3/users/cc2e08815035468f978939f8494711e5/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:29.646172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba8d88-84ee-472d-b3ff-97ecf7a2c10e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:29.675318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f76c46-ea2c-4953-a5e3-5406fc02f9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e66e4e650d342bd865bc2953a2e2549, user_id=980dfdeca6df4b32a0784a9bff691aa3, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:29.684952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f76c46-ea2c-4953-a5e3-5406fc02f9da admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:29.717249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 816/1636] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:29 2022] PUT /identity/v3/projects/1e66e4e650d342bd865bc2953a2e2549/users/980dfdeca6df4b32a0784a9bff691aa3/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:29.722233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9446be-4502-4413-a675-59655226172d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:29.722623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9446be-4502-4413-a675-59655226172d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:29.722988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9446be-4502-4413-a675-59655226172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:29.758294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ca9446be-4502-4413-a675-59655226172d None None] MFA Rules not processed for user `cc2e08815035468f978939f8494711e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:29.759645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ca9446be-4502-4413-a675-59655226172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:29.781658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:29.860941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba8d88-84ee-472d-b3ff-97ecf7a2c10e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:29.942173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 817/1637] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:29.944402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9446be-4502-4413-a675-59655226172d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:30.010854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62d83a-e10f-41ae-912e-4703f2b3204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:30.011582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62d83a-e10f-41ae-912e-4703f2b3204c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:30.011872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62d83a-e10f-41ae-912e-4703f2b3204c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:30.012136 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62d83a-e10f-41ae-912e-4703f2b3204c admin admin] PATH_INFO: `/v3/projects/82023d5e0854497b85b106a8dc30c1b8/users/73a4f8d757384d1381604f1158992a23/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:30.015586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62d83a-e10f-41ae-912e-4703f2b3204c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82023d5e0854497b85b106a8dc30c1b8, user_id=73a4f8d757384d1381604f1158992a23, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:30.019414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62d83a-e10f-41ae-912e-4703f2b3204c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:30.046849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 821/1638] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:29 2022] PUT /identity/v3/projects/82023d5e0854497b85b106a8dc30c1b8/users/73a4f8d757384d1381604f1158992a23/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:30.047659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62d83a-e10f-41ae-912e-4703f2b3204c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:30.068368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:30.091948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9446be-4502-4413-a675-59655226172d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:30.174280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62d83a-e10f-41ae-912e-4703f2b3204c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:30.184796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455a1b-4012-4cfb-8c4c-a477f147faa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:30.185887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a1b-4012-4cfb-8c4c-a477f147faa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:30.187881 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a1b-4012-4cfb-8c4c-a477f147faa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:30.188232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a1b-4012-4cfb-8c4c-a477f147faa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:30.219678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041fefb-cd70-49e7-994f-da1c92b6dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:30.221964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041fefb-cd70-49e7-994f-da1c92b6dcda admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:30.222240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041fefb-cd70-49e7-994f-da1c92b6dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:30.222513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041fefb-cd70-49e7-994f-da1c92b6dcda admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:30.223202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041fefb-cd70-49e7-994f-da1c92b6dcda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:30.224868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041fefb-cd70-49e7-994f-da1c92b6dcda admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:30.250779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 818/1639] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:30.252519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac766e-9752-4b26-bf39-5afe24b7a569 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:30.257130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac766e-9752-4b26-bf39-5afe24b7a569 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:30.257851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac766e-9752-4b26-bf39-5afe24b7a569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:30.309227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b455a1b-4012-4cfb-8c4c-a477f147faa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:30.310922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b455a1b-4012-4cfb-8c4c-a477f147faa9 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:30.319920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b2ac766e-9752-4b26-bf39-5afe24b7a569 None None] MFA Rules not processed for user `73a4f8d757384d1381604f1158992a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:30.321346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b2ac766e-9752-4b26-bf39-5afe24b7a569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:30.326616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 822/1640] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:30.327805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455a1b-4012-4cfb-8c4c-a477f147faa9 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:30.354295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455a1b-4012-4cfb-8c4c-a477f147faa9 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:30.364243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6ed97-53d3-4256-a668-e05f4b287d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:30.365733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ed97-53d3-4256-a668-e05f4b287d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:30.371934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ed97-53d3-4256-a668-e05f4b287d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:30.372601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ed97-53d3-4256-a668-e05f4b287d71 admin admin] PATH_INFO: `/v3/projects/1e66e4e650d342bd865bc2953a2e2549/users/980dfdeca6df4b32a0784a9bff691aa3/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:30.375575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6ed97-53d3-4256-a668-e05f4b287d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e66e4e650d342bd865bc2953a2e2549, user_id=980dfdeca6df4b32a0784a9bff691aa3, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:30.380398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6ed97-53d3-4256-a668-e05f4b287d71 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:30.398778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 823/1641] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:30 2022] PUT /identity/v3/projects/1e66e4e650d342bd865bc2953a2e2549/users/980dfdeca6df4b32a0784a9bff691aa3/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:30.411086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade0438-0dd0-4e02-8db0-03588afc759f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:30.411403 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade0438-0dd0-4e02-8db0-03588afc759f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:30.411680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade0438-0dd0-4e02-8db0-03588afc759f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:30.442842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:30.519907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-dade0438-0dd0-4e02-8db0-03588afc759f None None] MFA Rules not processed for user `980dfdeca6df4b32a0784a9bff691aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:30.527868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dade0438-0dd0-4e02-8db0-03588afc759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:30.534433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 819/1642] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:30.562746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac766e-9752-4b26-bf39-5afe24b7a569 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:30.575745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac766e-9752-4b26-bf39-5afe24b7a569 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:30.724232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8d04f-b6b0-4f1b-90ca-41260d8eb25a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:30.725237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d04f-b6b0-4f1b-90ca-41260d8eb25a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:30.725702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d04f-b6b0-4f1b-90ca-41260d8eb25a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:30.726154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d04f-b6b0-4f1b-90ca-41260d8eb25a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:30.781910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 824/1643] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:18:30.787581 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade0438-0dd0-4e02-8db0-03588afc759f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:30.804711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade0438-0dd0-4e02-8db0-03588afc759f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:30.808819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0b422-c788-44cc-b06f-b210130fc9d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:30.809450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0b422-c788-44cc-b06f-b210130fc9d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:30.809723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0b422-c788-44cc-b06f-b210130fc9d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:30.810031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0b422-c788-44cc-b06f-b210130fc9d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:30.865245 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:30.879554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8d04f-b6b0-4f1b-90ca-41260d8eb25a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:30.881702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8d04f-b6b0-4f1b-90ca-41260d8eb25a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:30.902506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 820/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:18:30.933534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0b422-c788-44cc-b06f-b210130fc9d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:30.935067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0b422-c788-44cc-b06f-b210130fc9d7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:30.957783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 825/1645] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:30.979397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8d04f-b6b0-4f1b-90ca-41260d8eb25a service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:30.991385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8d04f-b6b0-4f1b-90ca-41260d8eb25a service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:31.005727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f48008-2263-4850-a8bf-3f92ae53365a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:31.006368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f48008-2263-4850-a8bf-3f92ae53365a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:31.006693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f48008-2263-4850-a8bf-3f92ae53365a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:31.006965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f48008-2263-4850-a8bf-3f92ae53365a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:31.026768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f48008-2263-4850-a8bf-3f92ae53365a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:31.028071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f48008-2263-4850-a8bf-3f92ae53365a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:31.054051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 821/1646] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:31.181351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6b3a-4fce-4cf4-bd54-4b7391a217b7 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:31.181728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6b3a-4fce-4cf4-bd54-4b7391a217b7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:31.182087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6b3a-4fce-4cf4-bd54-4b7391a217b7 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:31.184266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 826/1647] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 31 12:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:18:31.203685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442eed2-f25d-4004-a86c-af4a6eaa1ebc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:31.204047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442eed2-f25d-4004-a86c-af4a6eaa1ebc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:31.204367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442eed2-f25d-4004-a86c-af4a6eaa1ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:31.245682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c442eed2-f25d-4004-a86c-af4a6eaa1ebc None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:31.249218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c442eed2-f25d-4004-a86c-af4a6eaa1ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:31.275060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a6b3a-4fce-4cf4-bd54-4b7391a217b7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:31.290612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a6b3a-4fce-4cf4-bd54-4b7391a217b7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:31.298929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ec185-e36e-4235-a406-d7cc5dc07201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:31.299570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ec185-e36e-4235-a406-d7cc5dc07201 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:31.299842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ec185-e36e-4235-a406-d7cc5dc07201 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:31.300396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ec185-e36e-4235-a406-d7cc5dc07201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:31.457580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ec185-e36e-4235-a406-d7cc5dc07201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:31.458939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ec185-e36e-4235-a406-d7cc5dc07201 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:31.471058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 827/1648] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:18:31.488835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ec185-e36e-4235-a406-d7cc5dc07201 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:31.498302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 822/1649] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 31 12:18:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:31.504100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc62b-8c9f-4bb3-936c-128fcbb0d036 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:31.504568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc62b-8c9f-4bb3-936c-128fcbb0d036 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:31.506116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc62b-8c9f-4bb3-936c-128fcbb0d036 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:31.513387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 823/1650] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 31 12:18:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:18:31.520038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ec185-e36e-4235-a406-d7cc5dc07201 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:31.523859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fad1f-95b8-4e79-a252-5ad5db61634d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:31.529198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fad1f-95b8-4e79-a252-5ad5db61634d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:31.531209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fad1f-95b8-4e79-a252-5ad5db61634d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:31.537901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19042d0-0715-4158-b2dd-22bbb4ec3af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:31.538554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19042d0-0715-4158-b2dd-22bbb4ec3af2 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:31.538826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19042d0-0715-4158-b2dd-22bbb4ec3af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:31.541197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19042d0-0715-4158-b2dd-22bbb4ec3af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:31.568024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:31.575065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f36fad1f-95b8-4e79-a252-5ad5db61634d None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:31.579715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f36fad1f-95b8-4e79-a252-5ad5db61634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:31.624908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 824/1651] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:18:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:31.751225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19042d0-0715-4158-b2dd-22bbb4ec3af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:31.757508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19042d0-0715-4158-b2dd-22bbb4ec3af2 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:31.786259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 828/1652] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:32.096358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fad1f-95b8-4e79-a252-5ad5db61634d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:32.137931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fad1f-95b8-4e79-a252-5ad5db61634d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:32.151434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cc3a9-987c-4e21-aa8c-fc71d3355f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:32.153092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cc3a9-987c-4e21-aa8c-fc71d3355f80 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:32.153891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cc3a9-987c-4e21-aa8c-fc71d3355f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:32.157162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cc3a9-987c-4e21-aa8c-fc71d3355f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:32.205989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cc3a9-987c-4e21-aa8c-fc71d3355f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:32.209474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cc3a9-987c-4e21-aa8c-fc71d3355f80 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:32.242267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 825/1653] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:35.810645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b661a-fff8-467e-995d-e7e731c35d40 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:35.811171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b661a-fff8-467e-995d-e7e731c35d40 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:35.811171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b661a-fff8-467e-995d-e7e731c35d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:35.863654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6b7b661a-fff8-467e-995d-e7e731c35d40 None None] MFA Rules not processed for user `09b76f14426c4241aa49812f06f35cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:35.864884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6b7b661a-fff8-467e-995d-e7e731c35d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:35.942847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cc3a9-987c-4e21-aa8c-fc71d3355f80 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:36.003494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:36.026208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 829/1654] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:36.041429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b661a-fff8-467e-995d-e7e731c35d40 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:36.061310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b661a-fff8-467e-995d-e7e731c35d40 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:36.076917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e71c1-3ec0-4d3e-aed7-3c0f4b068d47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:36.077549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e71c1-3ec0-4d3e-aed7-3c0f4b068d47 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:36.077820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e71c1-3ec0-4d3e-aed7-3c0f4b068d47 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:36.078086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e71c1-3ec0-4d3e-aed7-3c0f4b068d47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:36.092686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e71c1-3ec0-4d3e-aed7-3c0f4b068d47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:36.094369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e71c1-3ec0-4d3e-aed7-3c0f4b068d47 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:36.104956 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 830/1655] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:36.135102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cc3a9-987c-4e21-aa8c-fc71d3355f80 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:36.156329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:36.156999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:36.157276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:36.157735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] PATH_INFO: `/v3/users/34cfab2f4eca48e880b8277897a0ce7b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:36.159306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=34cfab2f4eca48e880b8277897a0ce7b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:36.160175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: WARNING py.warnings [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:18:36.160175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: warnings.warn(msg) Aug 31 12:18:36.160175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: Aug 31 12:18:36.161387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:36.317728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34cfab2f4eca48e880b8277897a0ce7b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:36.319028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34cfab2f4eca48e880b8277897a0ce7b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:36.372487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34cfab2f4eca48e880b8277897a0ce7b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:36.399769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34cfab2f4eca48e880b8277897a0ce7b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:36.416199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34cfab2f4eca48e880b8277897a0ce7b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:36.429765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 826/1656] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:18:35 2022] DELETE /identity/v3/users/34cfab2f4eca48e880b8277897a0ce7b => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:18:36.431661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0eaf1-8b2d-4d1c-afa7-f900567898dc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:36.431947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0eaf1-8b2d-4d1c-afa7-f900567898dc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:36.432218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0eaf1-8b2d-4d1c-afa7-f900567898dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:36.468507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-87e0eaf1-8b2d-4d1c-afa7-f900567898dc None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:36.471801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-87e0eaf1-8b2d-4d1c-afa7-f900567898dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:36.675918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 831/1657] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:36.708846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:36.710446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:37.003933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50839114-2cef-4f0f-8f0c-30c31d4b47bc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:37.011574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c698092-5b9b-4c74-bf97-a5d48813cace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:37.012979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698092-5b9b-4c74-bf97-a5d48813cace service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:37.013504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698092-5b9b-4c74-bf97-a5d48813cace service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:37.014104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698092-5b9b-4c74-bf97-a5d48813cace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:37.042915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c698092-5b9b-4c74-bf97-a5d48813cace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:37.045515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c698092-5b9b-4c74-bf97-a5d48813cace service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:37.073821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 827/1658] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:37.819177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0eaf1-8b2d-4d1c-afa7-f900567898dc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:37.820889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-87e0eaf1-8b2d-4d1c-afa7-f900567898dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:37.859223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0eaf1-8b2d-4d1c-afa7-f900567898dc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:37.881928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:37.882744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:37.883022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:37.883333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] PATH_INFO: `/v3/projects/c4730ec62c464218b57cdfdd5ee8a36f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:37.896545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4730ec62c464218b57cdfdd5ee8a36f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:37.897648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: WARNING py.warnings [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 12:18:37.897648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: warnings.warn(msg) Aug 31 12:18:37.897648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: Aug 31 12:18:37.913248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:38.082023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4730ec62c464218b57cdfdd5ee8a36f', 'request_id': 'req-9936e356-b626-44f3-bfcd-6c265e5d90b0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.123782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4730ec62c464218b57cdfdd5ee8a36f', 'request_id': 'req-9936e356-b626-44f3-bfcd-6c265e5d90b0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.147983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4730ec62c464218b57cdfdd5ee8a36f', 'request_id': 'req-9936e356-b626-44f3-bfcd-6c265e5d90b0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.169843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] The token cache is being invalidate because project c4730ec62c464218b57cdfdd5ee8a36f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:18:38.171393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.172815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9936e356-b626-44f3-bfcd-6c265e5d90b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.180441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 832/1659] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:18:37 2022] DELETE /identity/v3/projects/c4730ec62c464218b57cdfdd5ee8a36f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:38.184393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c698092-5b9b-4c74-bf97-a5d48813cace service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:38.198752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c698092-5b9b-4c74-bf97-a5d48813cace service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:38.217923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:38.523422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:38.533320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:38.533807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:38.535009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] PATH_INFO: `/v3/users/73a4f8d757384d1381604f1158992a23` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:38.543628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a4f8d757384d1381604f1158992a23)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:38.550936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:38.780612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a4f8d757384d1381604f1158992a23'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.784164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a4f8d757384d1381604f1158992a23'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.836587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a4f8d757384d1381604f1158992a23'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.871505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a4f8d757384d1381604f1158992a23'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.901491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a4f8d757384d1381604f1158992a23'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:38.923173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 828/1660] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:18:38 2022] DELETE /identity/v3/users/73a4f8d757384d1381604f1158992a23 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:40.291873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a185a-b722-496f-83f1-87c9a8b5e8e4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:40.292782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a185a-b722-496f-83f1-87c9a8b5e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:40.293208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a185a-b722-496f-83f1-87c9a8b5e8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:40.326216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d83a185a-b722-496f-83f1-87c9a8b5e8e4 None None] MFA Rules not processed for user `09b76f14426c4241aa49812f06f35cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:40.328863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d83a185a-b722-496f-83f1-87c9a8b5e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:40.425734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 833/1661] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:40.648579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:40.650684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:40.826075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645efec4-88b3-49df-9abc-bab7c0d24b3b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:40.833983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:40.835267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:40.835703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:40.836167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] PATH_INFO: `/v3/projects/82023d5e0854497b85b106a8dc30c1b8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:40.837495 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=82023d5e0854497b85b106a8dc30c1b8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:40.839812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:40.945328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:40.982406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82023d5e0854497b85b106a8dc30c1b8', 'request_id': 'req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:40.993198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82023d5e0854497b85b106a8dc30c1b8', 'request_id': 'req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:41.005016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82023d5e0854497b85b106a8dc30c1b8', 'request_id': 'req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:41.028649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] The token cache is being invalidate because project 82023d5e0854497b85b106a8dc30c1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:18:41.029270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:41.030326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-79b1edd7-aa5f-4ff1-9d21-1f5b55bca82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:41.035887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 829/1662] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:18:40 2022] DELETE /identity/v3/projects/82023d5e0854497b85b106a8dc30c1b8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:41.037590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c07c-097b-4a5c-82d7-59bb54aa4f21 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.037879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c07c-097b-4a5c-82d7-59bb54aa4f21 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.038174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c07c-097b-4a5c-82d7-59bb54aa4f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.063916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-98d5c07c-097b-4a5c-82d7-59bb54aa4f21 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:41.066374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-98d5c07c-097b-4a5c-82d7-59bb54aa4f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:41.072274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:41.177550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 834/1663] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:41.179222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02690c-c6b4-408d-8545-b44d35319a5e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.179796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02690c-c6b4-408d-8545-b44d35319a5e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.180068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02690c-c6b4-408d-8545-b44d35319a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.219678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2f02690c-c6b4-408d-8545-b44d35319a5e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:41.220987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2f02690c-c6b4-408d-8545-b44d35319a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:41.237894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c07c-097b-4a5c-82d7-59bb54aa4f21 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:41.250520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c07c-097b-4a5c-82d7-59bb54aa4f21 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:41.251500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 830/1664] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:18:41.278018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:41.278668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.278945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.279541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.280207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:41.281098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:41.282738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:41.398109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 835/1665] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:41.404828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02690c-c6b4-408d-8545-b44d35319a5e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:41.428015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02690c-c6b4-408d-8545-b44d35319a5e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:41.449540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8246e8a-278a-4de5-8eb5-f5033d9b7f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:41.453441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8246e8a-278a-4de5-8eb5-f5033d9b7f9b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.453726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8246e8a-278a-4de5-8eb5-f5033d9b7f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.459998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8246e8a-278a-4de5-8eb5-f5033d9b7f9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.463791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8246e8a-278a-4de5-8eb5-f5033d9b7f9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:41.465451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8246e8a-278a-4de5-8eb5-f5033d9b7f9b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:41.547318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 831/1666] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:41 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:41.554669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:41.580521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbb88-04e9-425d-98b4-e18fedd87c48 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:41.667112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81549db-c463-4c14-a0ec-898acd08511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:41.667957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81549db-c463-4c14-a0ec-898acd08511f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.668232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81549db-c463-4c14-a0ec-898acd08511f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.668502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81549db-c463-4c14-a0ec-898acd08511f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.669149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81549db-c463-4c14-a0ec-898acd08511f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:41.672317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81549db-c463-4c14-a0ec-898acd08511f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:41.729396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8246e8a-278a-4de5-8eb5-f5033d9b7f9b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:41.730368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 836/1667] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:41 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:41.757805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8246e8a-278a-4de5-8eb5-f5033d9b7f9b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:41.772562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be4845-5aa8-4472-8b8d-18bda2e09dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:41.777230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4845-5aa8-4472-8b8d-18bda2e09dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.778779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4845-5aa8-4472-8b8d-18bda2e09dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.779177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4845-5aa8-4472-8b8d-18bda2e09dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.782007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be4845-5aa8-4472-8b8d-18bda2e09dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:41.785556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be4845-5aa8-4472-8b8d-18bda2e09dc5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:41.809202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 832/1668] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:41.810063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81549db-c463-4c14-a0ec-898acd08511f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:41.824533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81549db-c463-4c14-a0ec-898acd08511f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:41.833777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5e615-c188-411f-bd76-a113561f26bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:41.834651 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5e615-c188-411f-bd76-a113561f26bb admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.834966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5e615-c188-411f-bd76-a113561f26bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.835327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5e615-c188-411f-bd76-a113561f26bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.837660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5e615-c188-411f-bd76-a113561f26bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:41.839483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5e615-c188-411f-bd76-a113561f26bb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:41.869654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 837/1669] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:41.871289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896159ba-f131-4797-8c1b-dea6ba1c4090 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.872439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896159ba-f131-4797-8c1b-dea6ba1c4090 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.872985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896159ba-f131-4797-8c1b-dea6ba1c4090 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.876565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5e615-c188-411f-bd76-a113561f26bb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:41.877173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 833/1670] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 31 12:18:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:18:41.891534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5e615-c188-411f-bd76-a113561f26bb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:41.894509 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69e134-6117-45c6-be16-5853751d2591 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.898390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69e134-6117-45c6-be16-5853751d2591 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.898856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69e134-6117-45c6-be16-5853751d2591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.900331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3704266-327b-4e06-8858-c45f9665819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:41.901118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3704266-327b-4e06-8858-c45f9665819c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:41.901462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3704266-327b-4e06-8858-c45f9665819c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:41.901803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3704266-327b-4e06-8858-c45f9665819c admin admin] PATH_INFO: `/v3/projects/fa2b2d0349af490e88cd539f7a0a2f92/users/038df7efeb0f4291912c0040ec90deb0/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:41.959415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3704266-327b-4e06-8858-c45f9665819c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2b2d0349af490e88cd539f7a0a2f92, user_id=038df7efeb0f4291912c0040ec90deb0, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:41.966217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3704266-327b-4e06-8858-c45f9665819c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:41.981486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ba69e134-6117-45c6-be16-5853751d2591 None None] MFA Rules not processed for user `ba8c7348ad2e457e80e1fdd2d93aacc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:41.984104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ba69e134-6117-45c6-be16-5853751d2591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:41.998533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 838/1671] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:41 2022] PUT /identity/v3/projects/fa2b2d0349af490e88cd539f7a0a2f92/users/038df7efeb0f4291912c0040ec90deb0/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:42.009314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4a29d-aa81-4e3e-89f7-a042a5fa2378 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:42.010251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4a29d-aa81-4e3e-89f7-a042a5fa2378 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:42.010731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4a29d-aa81-4e3e-89f7-a042a5fa2378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:42.057674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4ec4a29d-aa81-4e3e-89f7-a042a5fa2378 None None] MFA Rules not processed for user `038df7efeb0f4291912c0040ec90deb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:42.059908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ec4a29d-aa81-4e3e-89f7-a042a5fa2378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:42.135573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 834/1672] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:18:42.142448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:42.197517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69e134-6117-45c6-be16-5853751d2591 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:42.202129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ba69e134-6117-45c6-be16-5853751d2591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:42.222888 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69e134-6117-45c6-be16-5853751d2591 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:42.239964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6530a-9c6b-4413-bbe8-4ce16d3aa171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:42.240595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6530a-9c6b-4413-bbe8-4ce16d3aa171 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:42.240875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6530a-9c6b-4413-bbe8-4ce16d3aa171 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:42.241147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6530a-9c6b-4413-bbe8-4ce16d3aa171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:42.262291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6530a-9c6b-4413-bbe8-4ce16d3aa171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:42.264344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6530a-9c6b-4413-bbe8-4ce16d3aa171 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:42.282498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 835/1673] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:18:42.296476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 839/1674] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:42.301589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6530a-9c6b-4413-bbe8-4ce16d3aa171 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:42.453899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6530a-9c6b-4413-bbe8-4ce16d3aa171 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:42.485535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3860cf-754f-42ac-9ddd-defe49b09bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:42.488387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3860cf-754f-42ac-9ddd-defe49b09bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:42.490187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3860cf-754f-42ac-9ddd-defe49b09bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:42.490675 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3860cf-754f-42ac-9ddd-defe49b09bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:42.496516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3860cf-754f-42ac-9ddd-defe49b09bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:42.502120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3860cf-754f-42ac-9ddd-defe49b09bdd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:42.527510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:42.626187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 836/1675] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:42 2022] POST /identity/v3/projects => generated 377 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:42.630497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4a29d-aa81-4e3e-89f7-a042a5fa2378 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:42.654898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4a29d-aa81-4e3e-89f7-a042a5fa2378 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:42.897897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:42.899294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:42.901584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:42.902361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:42.905050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:42.909024 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:43.016398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 840/1676] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:42 2022] POST /identity/v3/users => generated 295 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:43.019380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3860cf-754f-42ac-9ddd-defe49b09bdd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:43.058747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3860cf-754f-42ac-9ddd-defe49b09bdd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:43.088667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:43.095782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:43.099736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31211709-978f-4006-943b-528efc1f1994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:43.103026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31211709-978f-4006-943b-528efc1f1994 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:43.103425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31211709-978f-4006-943b-528efc1f1994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:43.107065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31211709-978f-4006-943b-528efc1f1994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:43.108422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31211709-978f-4006-943b-528efc1f1994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:43.117041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31211709-978f-4006-943b-528efc1f1994 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:43.119361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127973-a4c9-47f4-8531-f6e2da1cef6c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:43.152500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:43.160682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:43.162424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:43.163784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] PATH_INFO: `/v3/users/fdc5e43a88d04557b81c95ce3e22e5a7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:43.166192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdc5e43a88d04557b81c95ce3e22e5a7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:43.167323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 837/1677] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:43.171772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31211709-978f-4006-943b-528efc1f1994 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:43.172868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:43.207458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31211709-978f-4006-943b-528efc1f1994 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:43.233946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e171fbb-333c-4968-b150-1f4f3a8ac314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:43.234724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e171fbb-333c-4968-b150-1f4f3a8ac314 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:43.237387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e171fbb-333c-4968-b150-1f4f3a8ac314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:43.239441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e171fbb-333c-4968-b150-1f4f3a8ac314 admin admin] PATH_INFO: `/v3/projects/792bd1fe023b42e5a94f36c7d47b0889/users/b7c5b2d188914ffcaa9d7fc2ff491181/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:43.285556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e171fbb-333c-4968-b150-1f4f3a8ac314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792bd1fe023b42e5a94f36c7d47b0889, user_id=b7c5b2d188914ffcaa9d7fc2ff491181, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:43.288490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e171fbb-333c-4968-b150-1f4f3a8ac314 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:43.306584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdc5e43a88d04557b81c95ce3e22e5a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:43.308766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdc5e43a88d04557b81c95ce3e22e5a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:43.322262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 838/1678] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:43 2022] PUT /identity/v3/projects/792bd1fe023b42e5a94f36c7d47b0889/users/b7c5b2d188914ffcaa9d7fc2ff491181/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:43.324039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e171fbb-333c-4968-b150-1f4f3a8ac314 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:43.329751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdc5e43a88d04557b81c95ce3e22e5a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:43.331163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdc5e43a88d04557b81c95ce3e22e5a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:43.358382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d10fa6e-db31-4dd4-bad6-227f4c123364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdc5e43a88d04557b81c95ce3e22e5a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:43.405093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 841/1679] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:18:43 2022] DELETE /identity/v3/users/fdc5e43a88d04557b81c95ce3e22e5a7 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:43.412118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b61b9-2be6-441a-9743-633be6f98dfc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:43.412965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b61b9-2be6-441a-9743-633be6f98dfc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:43.416827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b61b9-2be6-441a-9743-633be6f98dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:43.482199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4f0b61b9-2be6-441a-9743-633be6f98dfc None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:43.486095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4f0b61b9-2be6-441a-9743-633be6f98dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:43.553811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e171fbb-333c-4968-b150-1f4f3a8ac314 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:43.639781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:43.700501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 842/1680] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:43.745405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:43.748350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:43.748639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:43.748911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:43.752143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:43.753863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:43.785969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 839/1681] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:43.787929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b61b9-2be6-441a-9743-633be6f98dfc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:43.789463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4f0b61b9-2be6-441a-9743-633be6f98dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:43.800528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:43.802789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:43.810933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b61b9-2be6-441a-9743-633be6f98dfc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:43.823224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba21be0-4f0d-4f0f-8f66-de18041560db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:43.824070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21be0-4f0d-4f0f-8f66-de18041560db admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:43.824352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21be0-4f0d-4f0f-8f66-de18041560db admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:43.824626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21be0-4f0d-4f0f-8f66-de18041560db admin admin] PATH_INFO: `/v3/projects/792bd1fe023b42e5a94f36c7d47b0889/users/b7c5b2d188914ffcaa9d7fc2ff491181/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:43.836356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba21be0-4f0d-4f0f-8f66-de18041560db admin admin] RBAC: Authorizing `identity:create_grant(project_id=792bd1fe023b42e5a94f36c7d47b0889, user_id=b7c5b2d188914ffcaa9d7fc2ff491181, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:43.839373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba21be0-4f0d-4f0f-8f66-de18041560db admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:43.896502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 843/1682] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:43 2022] PUT /identity/v3/projects/792bd1fe023b42e5a94f36c7d47b0889/users/b7c5b2d188914ffcaa9d7fc2ff491181/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:43.899391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff80e3e-3099-4430-8211-a58d7704376d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:43.899676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff80e3e-3099-4430-8211-a58d7704376d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:43.899950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff80e3e-3099-4430-8211-a58d7704376d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:43.979782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b0c6-3aae-4d95-968b-c08a544b9c42 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:43.998831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:44.017521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bff80e3e-3099-4430-8211-a58d7704376d None None] MFA Rules not processed for user `b7c5b2d188914ffcaa9d7fc2ff491181`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:44.022345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bff80e3e-3099-4430-8211-a58d7704376d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:44.102664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:44.103455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:44.103730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:44.104004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:44.238588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 844/1683] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:18:44.253046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff80e3e-3099-4430-8211-a58d7704376d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:44.254703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bff80e3e-3099-4430-8211-a58d7704376d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:44.266925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:44.269099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:44.284164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 840/1684] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:44.405665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff80e3e-3099-4430-8211-a58d7704376d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:44.418149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:44.419135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:44.419836 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:44.420332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:44.424189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:44.562591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:44.566565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:44.578632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 845/1685] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:45.438114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:45.440608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:45.667661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb50a6-4bbf-4022-a1e4-ccc4c5a3440c service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:45.694408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bad492-4ab1-4e3f-ada1-d83200db9c35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:45.695396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bad492-4ab1-4e3f-ada1-d83200db9c35 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:45.695832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bad492-4ab1-4e3f-ada1-d83200db9c35 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:45.696250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bad492-4ab1-4e3f-ada1-d83200db9c35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:45.704773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-72bad492-4ab1-4e3f-ada1-d83200db9c35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:45.740454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bad492-4ab1-4e3f-ada1-d83200db9c35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:45.742641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bad492-4ab1-4e3f-ada1-d83200db9c35 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:45.795866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 841/1686] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:46.027736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:46.030412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:46.103656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a031545-95af-468b-b170-34da9b0e045d service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:46.116646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:46.120580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:46.121112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:46.133778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] PATH_INFO: `/v3/projects/27224908eebd4b759efd8c6b24bb32a9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:46.135950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27224908eebd4b759efd8c6b24bb32a9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:46.140924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:46.146094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:46.398979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27224908eebd4b759efd8c6b24bb32a9', 'request_id': 'req-45c033f4-7767-44dd-b169-e4e0c5cc04a1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:46.435498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27224908eebd4b759efd8c6b24bb32a9', 'request_id': 'req-45c033f4-7767-44dd-b169-e4e0c5cc04a1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:46.465815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27224908eebd4b759efd8c6b24bb32a9', 'request_id': 'req-45c033f4-7767-44dd-b169-e4e0c5cc04a1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:46.472712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c6586-9445-47b8-8abb-e688e326287c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:46.473203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c6586-9445-47b8-8abb-e688e326287c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:46.473302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c6586-9445-47b8-8abb-e688e326287c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:46.486640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] The token cache is being invalidate because project 27224908eebd4b759efd8c6b24bb32a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:18:46.487096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:46.488139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-45c033f4-7767-44dd-b169-e4e0c5cc04a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:46.493197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 846/1687] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:18:46 2022] DELETE /identity/v3/projects/27224908eebd4b759efd8c6b24bb32a9 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:46.495305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89316f0-9af4-4b14-bdbe-83b25265e78d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:46.495607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89316f0-9af4-4b14-bdbe-83b25265e78d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:46.496015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89316f0-9af4-4b14-bdbe-83b25265e78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:46.543998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ef6c6586-9445-47b8-8abb-e688e326287c None None] MFA Rules not processed for user `cace97d1adb44009acf2b7b093d1e885`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:46.545050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ef6c6586-9445-47b8-8abb-e688e326287c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:46.578081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f89316f0-9af4-4b14-bdbe-83b25265e78d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:46.579449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f89316f0-9af4-4b14-bdbe-83b25265e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:46.638288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:46.716781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 842/1688] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:46.719291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e08f7f-2d32-4d60-953e-48b9467f3302 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:46.719884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e08f7f-2d32-4d60-953e-48b9467f3302 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:46.721243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e08f7f-2d32-4d60-953e-48b9467f3302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:46.759718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 847/1689] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:46.767782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c863f-c068-4d4a-8bd2-d972d9cafdd8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:46.768210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c863f-c068-4d4a-8bd2-d972d9cafdd8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:46.768554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c863f-c068-4d4a-8bd2-d972d9cafdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:46.785273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-02e08f7f-2d32-4d60-953e-48b9467f3302 None None] MFA Rules not processed for user `cace97d1adb44009acf2b7b093d1e885`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:46.790357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-02e08f7f-2d32-4d60-953e-48b9467f3302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:46.820836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 843/1690] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:46.822728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e08f7f-2d32-4d60-953e-48b9467f3302 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:46.827889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-02e08f7f-2d32-4d60-953e-48b9467f3302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:46.847559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ca1c863f-c068-4d4a-8bd2-d972d9cafdd8 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:46.849108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ca1c863f-c068-4d4a-8bd2-d972d9cafdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:46.882624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 848/1691] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:46.887060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c863f-c068-4d4a-8bd2-d972d9cafdd8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:46.943491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c863f-c068-4d4a-8bd2-d972d9cafdd8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:46.969958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:46.972723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:46.973017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:46.973294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:46.974016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:46.974391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:46.981345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:47.046949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 849/1692] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:18:47.050856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:47.076683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e08f7f-2d32-4d60-953e-48b9467f3302 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:47.080196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fc1b2-8eb2-48b6-97c5-5bd1b9f74e2f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:47.093940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbbf7a-8e5b-40e4-b5b3-9bbb749989b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:47.095586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbbf7a-8e5b-40e4-b5b3-9bbb749989b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.096868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbbf7a-8e5b-40e4-b5b3-9bbb749989b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.097591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbbf7a-8e5b-40e4-b5b3-9bbb749989b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.098521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbbf7a-8e5b-40e4-b5b3-9bbb749989b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:47.099982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff2ab-f95b-4e9c-9d81-279133078914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:47.100630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff2ab-f95b-4e9c-9d81-279133078914 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.100979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff2ab-f95b-4e9c-9d81-279133078914 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.101157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff2ab-f95b-4e9c-9d81-279133078914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.107352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbbf7a-8e5b-40e4-b5b3-9bbb749989b0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:47.135191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ff2ab-f95b-4e9c-9d81-279133078914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:47.145090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ff2ab-f95b-4e9c-9d81-279133078914 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:47.184243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 844/1693] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:47.210503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 850/1694] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:47 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:47.211434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff2ab-f95b-4e9c-9d81-279133078914 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:47.229756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff2ab-f95b-4e9c-9d81-279133078914 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:47.451525 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:47.452211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.452687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.453033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.453789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:47.458954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:47.534573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbbf7a-8e5b-40e4-b5b3-9bbb749989b0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:47.535106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 845/1695] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:47 2022] POST /identity/v3/users => generated 287 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:47.562993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbbf7a-8e5b-40e4-b5b3-9bbb749989b0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:47.587076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a42a-a22e-41b1-bd04-c6cee59eae99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:47.590691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a42a-a22e-41b1-bd04-c6cee59eae99 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.590964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a42a-a22e-41b1-bd04-c6cee59eae99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.591257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503a42a-a22e-41b1-bd04-c6cee59eae99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.596416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503a42a-a22e-41b1-bd04-c6cee59eae99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:47.609488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503a42a-a22e-41b1-bd04-c6cee59eae99 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:47.638308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:47.640338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:47.659139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 851/1696] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:47.669071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a42a-a22e-41b1-bd04-c6cee59eae99 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:47.684858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1eec5-38a1-4cbe-88d1-d9f7bd51a640 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:47.699494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:47.701191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.701621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.702485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] PATH_INFO: `/v3/users/d22acbf59be246d487e31ba6400bb5ba` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.704351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22acbf59be246d487e31ba6400bb5ba)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:47.707308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:47.709872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503a42a-a22e-41b1-bd04-c6cee59eae99 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:47.731394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eca70a-84aa-40a6-9b94-76c5af654086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:47.733318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eca70a-84aa-40a6-9b94-76c5af654086 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.733600 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eca70a-84aa-40a6-9b94-76c5af654086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.733873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eca70a-84aa-40a6-9b94-76c5af654086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.734534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eca70a-84aa-40a6-9b94-76c5af654086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:47.736322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eca70a-84aa-40a6-9b94-76c5af654086 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:47.772919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 852/1697] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:47.774809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eca70a-84aa-40a6-9b94-76c5af654086 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:47.802767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eca70a-84aa-40a6-9b94-76c5af654086 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:47.804736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:47.809588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48cadf-6b9f-4da5-8f39-ca7138c16bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:47.811271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22acbf59be246d487e31ba6400bb5ba'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:47.811697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48cadf-6b9f-4da5-8f39-ca7138c16bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.812284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48cadf-6b9f-4da5-8f39-ca7138c16bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.812493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22acbf59be246d487e31ba6400bb5ba'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:47.812932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48cadf-6b9f-4da5-8f39-ca7138c16bf8 admin admin] PATH_INFO: `/v3/projects/86e3f044eac8427bbba518b6b2ef45eb/users/dc79a224290b4d9ba30aebe72086c21d/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.838933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48cadf-6b9f-4da5-8f39-ca7138c16bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e3f044eac8427bbba518b6b2ef45eb, user_id=dc79a224290b4d9ba30aebe72086c21d, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:47.842339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48cadf-6b9f-4da5-8f39-ca7138c16bf8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:47.851971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22acbf59be246d487e31ba6400bb5ba'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:47.865461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22acbf59be246d487e31ba6400bb5ba'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:47.867604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 853/1698] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:47 2022] PUT /identity/v3/projects/86e3f044eac8427bbba518b6b2ef45eb/users/dc79a224290b4d9ba30aebe72086c21d/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:47.880216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ab883-2b75-4c8a-86e5-61402cdd95dd None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.880718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ab883-2b75-4c8a-86e5-61402cdd95dd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.881302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ab883-2b75-4c8a-86e5-61402cdd95dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.884443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-caa3b50c-406c-478b-b68b-6caaf86a7cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22acbf59be246d487e31ba6400bb5ba'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:47.903589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 846/1699] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:18:47 2022] DELETE /identity/v3/users/d22acbf59be246d487e31ba6400bb5ba => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:47.904170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:47.906427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb59d5f-28f4-4d31-8adf-6e2268b9c3c1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:47.906714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb59d5f-28f4-4d31-8adf-6e2268b9c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:47.909675 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb59d5f-28f4-4d31-8adf-6e2268b9c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:47.937384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6cb59d5f-28f4-4d31-8adf-6e2268b9c3c1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:47.938559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-494ab883-2b75-4c8a-86e5-61402cdd95dd None None] MFA Rules not processed for user `dc79a224290b4d9ba30aebe72086c21d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:47.939264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6cb59d5f-28f4-4d31-8adf-6e2268b9c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:47.940411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-494ab883-2b75-4c8a-86e5-61402cdd95dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:48.038513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 847/1700] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:48.045743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb59d5f-28f4-4d31-8adf-6e2268b9c3c1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:48.047344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6cb59d5f-28f4-4d31-8adf-6e2268b9c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:48.066188 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 854/1701] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:48.070789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ab883-2b75-4c8a-86e5-61402cdd95dd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:48.072725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-494ab883-2b75-4c8a-86e5-61402cdd95dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:48.100135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ab883-2b75-4c8a-86e5-61402cdd95dd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:48.115668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513ea1d-3df3-4c63-a301-26dccb62faf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:48.118204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513ea1d-3df3-4c63-a301-26dccb62faf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:48.118828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513ea1d-3df3-4c63-a301-26dccb62faf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:48.119195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513ea1d-3df3-4c63-a301-26dccb62faf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:48.120031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513ea1d-3df3-4c63-a301-26dccb62faf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:48.122907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb59d5f-28f4-4d31-8adf-6e2268b9c3c1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:48.123095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513ea1d-3df3-4c63-a301-26dccb62faf2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:48.153696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ffa53-563e-46bf-8a82-02c41c3a760e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:48.154324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ffa53-563e-46bf-8a82-02c41c3a760e service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:48.154600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ffa53-563e-46bf-8a82-02c41c3a760e service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:48.154867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ffa53-563e-46bf-8a82-02c41c3a760e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:48.176435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ffa53-563e-46bf-8a82-02c41c3a760e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:48.177944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ffa53-563e-46bf-8a82-02c41c3a760e service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:48.198387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 855/1702] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:48 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:48.215564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513ea1d-3df3-4c63-a301-26dccb62faf2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:48.220400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 848/1703] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:48.320681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513ea1d-3df3-4c63-a301-26dccb62faf2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:48.335794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b07619-c09c-41e1-a906-5f03dd833d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:48.336887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07619-c09c-41e1-a906-5f03dd833d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:48.337946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07619-c09c-41e1-a906-5f03dd833d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:48.339690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07619-c09c-41e1-a906-5f03dd833d53 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:48.340685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b07619-c09c-41e1-a906-5f03dd833d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:48.345209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b07619-c09c-41e1-a906-5f03dd833d53 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:48.409053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 856/1704] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:48 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:48.413642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ffa53-563e-46bf-8a82-02c41c3a760e service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:48.428350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ffa53-563e-46bf-8a82-02c41c3a760e service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:48.435086 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbd5d5-e7be-403b-9c77-7266629b6eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:48.436271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbd5d5-e7be-403b-9c77-7266629b6eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:48.436413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbd5d5-e7be-403b-9c77-7266629b6eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:48.436893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbd5d5-e7be-403b-9c77-7266629b6eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:48.437684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbd5d5-e7be-403b-9c77-7266629b6eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:48.439673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbd5d5-e7be-403b-9c77-7266629b6eb1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:48.462268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 849/1705] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:48.463097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b07619-c09c-41e1-a906-5f03dd833d53 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:48.479006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b07619-c09c-41e1-a906-5f03dd833d53 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:48.484334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:48.485205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:48.485587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:48.485995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] PATH_INFO: `/v3/projects/303e900f1c1945deb2c3f472188f1e35/users/f2f6ecf739ac420eadaf1ddfb899fa10/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:48.515292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] RBAC: Authorizing `identity:create_grant(project_id=303e900f1c1945deb2c3f472188f1e35, user_id=f2f6ecf739ac420eadaf1ddfb899fa10, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:48.519109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:48.552470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbd5d5-e7be-403b-9c77-7266629b6eb1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:48.553575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 857/1706] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:48 2022] PUT /identity/v3/projects/303e900f1c1945deb2c3f472188f1e35/users/f2f6ecf739ac420eadaf1ddfb899fa10/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:48.566179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbd5d5-e7be-403b-9c77-7266629b6eb1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:48.656587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:48.659491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:48.663966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:48.691621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146c59f-5eac-4724-9d1f-41a87da6416e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:48.700909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146c59f-5eac-4724-9d1f-41a87da6416e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:48.700909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146c59f-5eac-4724-9d1f-41a87da6416e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:48.700909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146c59f-5eac-4724-9d1f-41a87da6416e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:48.700909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146c59f-5eac-4724-9d1f-41a87da6416e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:48.700909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146c59f-5eac-4724-9d1f-41a87da6416e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:48.717474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 850/1707] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:48.722162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146c59f-5eac-4724-9d1f-41a87da6416e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:48.734277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146c59f-5eac-4724-9d1f-41a87da6416e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:48.742226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9045f-fbaf-4e3b-a7b6-6279d74e7ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:48.743110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9045f-fbaf-4e3b-a7b6-6279d74e7ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:48.743512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9045f-fbaf-4e3b-a7b6-6279d74e7ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:48.743897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9045f-fbaf-4e3b-a7b6-6279d74e7ac2 admin admin] PATH_INFO: `/v3/projects/303e900f1c1945deb2c3f472188f1e35/users/f2f6ecf739ac420eadaf1ddfb899fa10/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:48.745977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9045f-fbaf-4e3b-a7b6-6279d74e7ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303e900f1c1945deb2c3f472188f1e35, user_id=f2f6ecf739ac420eadaf1ddfb899fa10, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:48.746472 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:48.751458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9045f-fbaf-4e3b-a7b6-6279d74e7ac2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:48.772022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 851/1708] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:48 2022] PUT /identity/v3/projects/303e900f1c1945deb2c3f472188f1e35/users/f2f6ecf739ac420eadaf1ddfb899fa10/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:18:48.786149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47728bf6-c94e-4650-a96e-c03da3162804 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:48.788080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47728bf6-c94e-4650-a96e-c03da3162804 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:48.788652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47728bf6-c94e-4650-a96e-c03da3162804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:48.842550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98658a-88cd-4585-b860-f653c85bc36f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:48.903540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-47728bf6-c94e-4650-a96e-c03da3162804 None None] MFA Rules not processed for user `f2f6ecf739ac420eadaf1ddfb899fa10`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:48.915030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-47728bf6-c94e-4650-a96e-c03da3162804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:49.034611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:49.035366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:49.035650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:49.035923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] PATH_INFO: `/v3/projects/1d7219a4c67b439ba225a0397c3c2505` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:49.037117 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7219a4c67b439ba225a0397c3c2505)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:49.039700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:49.102552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 852/1709] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:49.128605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7219a4c67b439ba225a0397c3c2505', 'request_id': 'req-95e0e916-870c-4150-b168-1a0b21c219fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.133247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47728bf6-c94e-4650-a96e-c03da3162804 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:49.134821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-47728bf6-c94e-4650-a96e-c03da3162804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:49.138330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7219a4c67b439ba225a0397c3c2505', 'request_id': 'req-95e0e916-870c-4150-b168-1a0b21c219fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.152885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7219a4c67b439ba225a0397c3c2505', 'request_id': 'req-95e0e916-870c-4150-b168-1a0b21c219fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.167438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] The token cache is being invalidate because project 1d7219a4c67b439ba225a0397c3c2505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:18:49.167977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.169625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.171886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 858/1710] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:18:48 2022] DELETE /identity/v3/projects/1d7219a4c67b439ba225a0397c3c2505 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:49.176992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:49.179059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:49.229628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47728bf6-c94e-4650-a96e-c03da3162804 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:49.351351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74585b96-67ef-4bde-94a7-4fcfec1b07da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:49.352385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74585b96-67ef-4bde-94a7-4fcfec1b07da service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:49.352859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74585b96-67ef-4bde-94a7-4fcfec1b07da service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:49.353469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0e916-870c-4150-b168-1a0b21c219fa admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:49.353857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74585b96-67ef-4bde-94a7-4fcfec1b07da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:49.357079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-74585b96-67ef-4bde-94a7-4fcfec1b07da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:49.364650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:49.365334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:49.365615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:49.365889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] PATH_INFO: `/v3/users/980dfdeca6df4b32a0784a9bff691aa3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:49.369914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=980dfdeca6df4b32a0784a9bff691aa3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:49.388512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:49.447835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:49.455154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:49.543437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980dfdeca6df4b32a0784a9bff691aa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.544511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980dfdeca6df4b32a0784a9bff691aa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.546659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74585b96-67ef-4bde-94a7-4fcfec1b07da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:49.551764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74585b96-67ef-4bde-94a7-4fcfec1b07da service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:49.564782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980dfdeca6df4b32a0784a9bff691aa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.566265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980dfdeca6df4b32a0784a9bff691aa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.602673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980dfdeca6df4b32a0784a9bff691aa3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:49.608230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 853/1711] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:18:49.609849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d8c0d-713f-4b88-9775-6cc3243d9dff None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:49.610116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d8c0d-713f-4b88-9775-6cc3243d9dff None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:49.610402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d8c0d-713f-4b88-9775-6cc3243d9dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:49.698552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 859/1712] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:18:49 2022] DELETE /identity/v3/users/980dfdeca6df4b32a0784a9bff691aa3 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:49.719746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7e5d8c0d-713f-4b88-9775-6cc3243d9dff None None] MFA Rules not processed for user `a8320be5e9644b6699ac526913e109eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:49.721461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7e5d8c0d-713f-4b88-9775-6cc3243d9dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:49.754911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:49.757212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:49.912815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 854/1713] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:49.921822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6d1f-d928-43b1-bc5b-54d172c2762d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:49.921822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6d1f-d928-43b1-bc5b-54d172c2762d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:49.921822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6d1f-d928-43b1-bc5b-54d172c2762d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:49.922067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b92b9-e007-4585-9e5b-ad61051c2ee6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:49.932888 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:49.941958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:49.953840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:49.954363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:49.960860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:49.985007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4b1a6d1f-d928-43b1-bc5b-54d172c2762d None None] MFA Rules not processed for user `4c3e5c1776b8470a9b677bac8dfbc9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:49.993088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4b1a6d1f-d928-43b1-bc5b-54d172c2762d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:50.244979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 855/1714] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:50.246463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf36c2-e668-4520-a4b2-6b1cb76ff6c8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:50.246756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf36c2-e668-4520-a4b2-6b1cb76ff6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:50.247031 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf36c2-e668-4520-a4b2-6b1cb76ff6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:50.265298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:50.270787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:50.300230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 860/1715] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:18:50.301793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:50.309292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:50.320550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cdbf36c2-e668-4520-a4b2-6b1cb76ff6c8 None None] MFA Rules not processed for user `a8320be5e9644b6699ac526913e109eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:50.327705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cdbf36c2-e668-4520-a4b2-6b1cb76ff6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:50.382168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 856/1716] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:50.427390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf36c2-e668-4520-a4b2-6b1cb76ff6c8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:50.429557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cdbf36c2-e668-4520-a4b2-6b1cb76ff6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:50.473847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:50.519404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f6b2e-742c-4d0e-bba3-11fceed4b363 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:50.535974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:50.538698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:50.539397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:50.539829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:50.553242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:50.555036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:50.569583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 861/1717] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:50.570993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:50.572545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:50.586708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de66d-a724-4188-810f-1d5226f62bca service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:50.590591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:50.592182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:50.592508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:50.599884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:50.607195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:50.665390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:50.796462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:50.802075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:50.838891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 862/1718] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:50.863087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:50.867596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:50.883841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf36c2-e668-4520-a4b2-6b1cb76ff6c8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:50.903332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45949c13-60ac-43e7-8dfe-df3785a8aa66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:50.910168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45949c13-60ac-43e7-8dfe-df3785a8aa66 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:50.910686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45949c13-60ac-43e7-8dfe-df3785a8aa66 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:50.910968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45949c13-60ac-43e7-8dfe-df3785a8aa66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:50.923396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45949c13-60ac-43e7-8dfe-df3785a8aa66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:50.933268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45949c13-60ac-43e7-8dfe-df3785a8aa66 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:50.945419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 857/1719] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:18:51.163821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4506a9e-2e05-4f8e-b3e4-2410a504e548 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:51.180290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:51.180962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:51.182588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:51.182886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] PATH_INFO: `/v3/projects/1e66e4e650d342bd865bc2953a2e2549` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:51.185974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e66e4e650d342bd865bc2953a2e2549)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:51.194630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:51.363077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e66e4e650d342bd865bc2953a2e2549', 'request_id': 'req-aa8e0bcf-3b25-4235-9993-16e945dc28fd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:51.396880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e66e4e650d342bd865bc2953a2e2549', 'request_id': 'req-aa8e0bcf-3b25-4235-9993-16e945dc28fd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:51.419808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e66e4e650d342bd865bc2953a2e2549', 'request_id': 'req-aa8e0bcf-3b25-4235-9993-16e945dc28fd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:51.431050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:51.444131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] The token cache is being invalidate because project 1e66e4e650d342bd865bc2953a2e2549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:18:51.444492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:51.448728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa8e0bcf-3b25-4235-9993-16e945dc28fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:51.452159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 863/1720] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:18:50 2022] DELETE /identity/v3/projects/1e66e4e650d342bd865bc2953a2e2549 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:51.457667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42f65c-1dfb-4bbd-9cd9-f1e4811e5377 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:51.458436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42f65c-1dfb-4bbd-9cd9-f1e4811e5377 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:51.458864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42f65c-1dfb-4bbd-9cd9-f1e4811e5377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:51.517119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9d42f65c-1dfb-4bbd-9cd9-f1e4811e5377 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:51.521648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9d42f65c-1dfb-4bbd-9cd9-f1e4811e5377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:51.771302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 858/1721] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:51.781264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e332c-b41a-486e-a2a2-ad550e73a605 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:51.781791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e332c-b41a-486e-a2a2-ad550e73a605 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:51.782246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e332c-b41a-486e-a2a2-ad550e73a605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:51.845159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1a5e332c-b41a-486e-a2a2-ad550e73a605 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:51.845159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1a5e332c-b41a-486e-a2a2-ad550e73a605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:51.855282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 864/1722] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:51.857573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42f65c-1dfb-4bbd-9cd9-f1e4811e5377 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:51.874985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42f65c-1dfb-4bbd-9cd9-f1e4811e5377 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:51.886604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:51.888013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:51.888455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:51.889175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:51.896561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:18:51.898270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:51.904410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:51.940670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 859/1723] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:18:51.943909 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e332c-b41a-486e-a2a2-ad550e73a605 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:51.955810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e332c-b41a-486e-a2a2-ad550e73a605 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:51.969760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987db034-513f-4ee4-96b8-0cc33972815a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:51.972954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987db034-513f-4ee4-96b8-0cc33972815a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:51.973648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987db034-513f-4ee4-96b8-0cc33972815a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:51.974587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987db034-513f-4ee4-96b8-0cc33972815a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:51.978392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987db034-513f-4ee4-96b8-0cc33972815a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:51.980790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987db034-513f-4ee4-96b8-0cc33972815a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:52.076394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 865/1724] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:51 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:52.082261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:52.100687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ac3d4-64c2-4622-bdaf-789d7912b2d1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:52.206755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a46de-1fd3-420c-992a-9855b597ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:52.207554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a46de-1fd3-420c-992a-9855b597ba9e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:52.207974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a46de-1fd3-420c-992a-9855b597ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:52.208414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a46de-1fd3-420c-992a-9855b597ba9e admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:52.209152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a46de-1fd3-420c-992a-9855b597ba9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:52.220124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a46de-1fd3-420c-992a-9855b597ba9e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:52.267012 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 860/1725] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:52 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:18:52.268620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987db034-513f-4ee4-96b8-0cc33972815a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:52.286464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987db034-513f-4ee4-96b8-0cc33972815a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:52.304251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6f88e-886c-416d-b575-60fbc7263e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:52.305386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6f88e-886c-416d-b575-60fbc7263e9b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:52.305748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6f88e-886c-416d-b575-60fbc7263e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:52.306480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6f88e-886c-416d-b575-60fbc7263e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:52.307286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a6f88e-886c-416d-b575-60fbc7263e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:52.311222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a6f88e-886c-416d-b575-60fbc7263e9b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:52.342252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 866/1726] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:52.345098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a46de-1fd3-420c-992a-9855b597ba9e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:52.363641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a46de-1fd3-420c-992a-9855b597ba9e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:52.375728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01efe77-8613-4890-b82d-2664ff84d099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:52.376360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01efe77-8613-4890-b82d-2664ff84d099 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:52.376769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01efe77-8613-4890-b82d-2664ff84d099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:52.377273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01efe77-8613-4890-b82d-2664ff84d099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:52.377988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01efe77-8613-4890-b82d-2664ff84d099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:52.380607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01efe77-8613-4890-b82d-2664ff84d099 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:52.408243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 861/1727] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:18:52.413369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6f88e-886c-416d-b575-60fbc7263e9b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:52.433073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6f88e-886c-416d-b575-60fbc7263e9b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:52.446376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c421bf6-720c-44ab-837e-086a1239869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:52.447265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421bf6-720c-44ab-837e-086a1239869d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:52.448955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421bf6-720c-44ab-837e-086a1239869d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:52.449781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421bf6-720c-44ab-837e-086a1239869d admin admin] PATH_INFO: `/v3/projects/4ef29c50c368400780306d7344d0f15a/users/32e37b53d5844aa48e812855cef6c77f/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:52.493115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c421bf6-720c-44ab-837e-086a1239869d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef29c50c368400780306d7344d0f15a, user_id=32e37b53d5844aa48e812855cef6c77f, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:52.495876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c421bf6-720c-44ab-837e-086a1239869d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:52.519275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 867/1728] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:52 2022] PUT /identity/v3/projects/4ef29c50c368400780306d7344d0f15a/users/32e37b53d5844aa48e812855cef6c77f/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:52.525124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950581f4-1920-4be4-ac95-f6fcf44e77bd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:52.526378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950581f4-1920-4be4-ac95-f6fcf44e77bd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:52.526745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950581f4-1920-4be4-ac95-f6fcf44e77bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:52.588491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-950581f4-1920-4be4-ac95-f6fcf44e77bd None None] MFA Rules not processed for user `32e37b53d5844aa48e812855cef6c77f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:52.590039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-950581f4-1920-4be4-ac95-f6fcf44e77bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:52.655566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:52.781690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 862/1729] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:52.785591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c421bf6-720c-44ab-837e-086a1239869d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:52.994004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493f8a-4408-4eaf-b3fb-2b92bed251d2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:52.994292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493f8a-4408-4eaf-b3fb-2b92bed251d2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:52.994583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493f8a-4408-4eaf-b3fb-2b92bed251d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:53.062410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cb493f8a-4408-4eaf-b3fb-2b92bed251d2 None None] MFA Rules not processed for user `a8320be5e9644b6699ac526913e109eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:53.063903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cb493f8a-4408-4eaf-b3fb-2b92bed251d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:53.128717 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c421bf6-720c-44ab-837e-086a1239869d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:53.153068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167e862-e18e-44cc-9eb8-a51da18e76bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:53.155219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e862-e18e-44cc-9eb8-a51da18e76bf admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:53.156408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e862-e18e-44cc-9eb8-a51da18e76bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:53.157434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e862-e18e-44cc-9eb8-a51da18e76bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:53.158583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167e862-e18e-44cc-9eb8-a51da18e76bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:53.162771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167e862-e18e-44cc-9eb8-a51da18e76bf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:53.237053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 868/1730] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:18:52 2022] POST /identity/v3/projects => generated 369 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:53.238147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167e862-e18e-44cc-9eb8-a51da18e76bf admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:53.303477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 863/1731] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:18:53.402944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167e862-e18e-44cc-9eb8-a51da18e76bf admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:53.408543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62f24e-2bf3-4798-87ca-d8a85e83232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:53.409184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62f24e-2bf3-4798-87ca-d8a85e83232e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:53.409457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62f24e-2bf3-4798-87ca-d8a85e83232e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:53.409729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62f24e-2bf3-4798-87ca-d8a85e83232e admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:53.410373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62f24e-2bf3-4798-87ca-d8a85e83232e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:53.412378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62f24e-2bf3-4798-87ca-d8a85e83232e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:53.500281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 869/1732] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:18:53 2022] POST /identity/v3/users => generated 291 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:18:53.505188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb493f8a-4408-4eaf-b3fb-2b92bed251d2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:53.530066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb493f8a-4408-4eaf-b3fb-2b92bed251d2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:53.538127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0453f-87cc-4156-b2dd-5bd2091f5296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:53.539114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0453f-87cc-4156-b2dd-5bd2091f5296 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:53.539841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0453f-87cc-4156-b2dd-5bd2091f5296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:53.540349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0453f-87cc-4156-b2dd-5bd2091f5296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:53.541213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0453f-87cc-4156-b2dd-5bd2091f5296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:53.543553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0453f-87cc-4156-b2dd-5bd2091f5296 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:53.563960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 864/1733] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:53.566707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62f24e-2bf3-4798-87ca-d8a85e83232e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:53.585043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62f24e-2bf3-4798-87ca-d8a85e83232e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:53.592701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1fe2c-7f78-4c69-8b4d-bb3e80d10c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:53.593706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1fe2c-7f78-4c69-8b4d-bb3e80d10c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:53.594105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1fe2c-7f78-4c69-8b4d-bb3e80d10c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:53.594476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1fe2c-7f78-4c69-8b4d-bb3e80d10c9a admin admin] PATH_INFO: `/v3/projects/d935e3ec3875494abe1c5cd68ad5084b/users/d84c43e952924fe5bd3392fd2d14aeb6/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:53.600751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:53.623533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1fe2c-7f78-4c69-8b4d-bb3e80d10c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d935e3ec3875494abe1c5cd68ad5084b, user_id=d84c43e952924fe5bd3392fd2d14aeb6, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:53.627966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1fe2c-7f78-4c69-8b4d-bb3e80d10c9a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:53.664187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 870/1734] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:53 2022] PUT /identity/v3/projects/d935e3ec3875494abe1c5cd68ad5084b/users/d84c43e952924fe5bd3392fd2d14aeb6/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:53.666163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0453f-87cc-4156-b2dd-5bd2091f5296 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:53.690416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0453f-87cc-4156-b2dd-5bd2091f5296 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:53.894435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630bf9d-4e71-45ef-aa81-9aa70abe10b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:53.896203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630bf9d-4e71-45ef-aa81-9aa70abe10b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:53.896609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630bf9d-4e71-45ef-aa81-9aa70abe10b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:53.897018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630bf9d-4e71-45ef-aa81-9aa70abe10b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:53.897960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630bf9d-4e71-45ef-aa81-9aa70abe10b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:53.900098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630bf9d-4e71-45ef-aa81-9aa70abe10b1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:53.938430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 865/1735] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:18:53.942955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1fe2c-7f78-4c69-8b4d-bb3e80d10c9a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:53.958048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1fe2c-7f78-4c69-8b4d-bb3e80d10c9a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:53.976120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004b9ae-82b5-4a0d-b64b-5022af37852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:53.976639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b9ae-82b5-4a0d-b64b-5022af37852c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:53.977218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b9ae-82b5-4a0d-b64b-5022af37852c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:53.977625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b9ae-82b5-4a0d-b64b-5022af37852c admin admin] PATH_INFO: `/v3/projects/d935e3ec3875494abe1c5cd68ad5084b/users/d84c43e952924fe5bd3392fd2d14aeb6/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:53.980053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004b9ae-82b5-4a0d-b64b-5022af37852c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d935e3ec3875494abe1c5cd68ad5084b, user_id=d84c43e952924fe5bd3392fd2d14aeb6, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:53.983664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004b9ae-82b5-4a0d-b64b-5022af37852c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:54.001252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 871/1736] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:18:53 2022] PUT /identity/v3/projects/d935e3ec3875494abe1c5cd68ad5084b/users/d84c43e952924fe5bd3392fd2d14aeb6/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:54.012261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2c3a-6e54-4551-92c7-6445f1bcf2bc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:54.012615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2c3a-6e54-4551-92c7-6445f1bcf2bc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:54.012953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2c3a-6e54-4551-92c7-6445f1bcf2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:54.059886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4fc7-d86d-4b93-8561-2d8bbc28e272 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:54.059886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4fc7-d86d-4b93-8561-2d8bbc28e272 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:54.059886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4fc7-d86d-4b93-8561-2d8bbc28e272 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:54.061509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 872/1737] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 31 12:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 12:18:54.068991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115d632-e38e-4bb3-b481-5dce2c5c90b7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:54.069626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115d632-e38e-4bb3-b481-5dce2c5c90b7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:54.070061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115d632-e38e-4bb3-b481-5dce2c5c90b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:54.087128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a92e2c3a-6e54-4551-92c7-6445f1bcf2bc None None] MFA Rules not processed for user `d84c43e952924fe5bd3392fd2d14aeb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:54.088521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a92e2c3a-6e54-4551-92c7-6445f1bcf2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:54.097417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4115d632-e38e-4bb3-b481-5dce2c5c90b7 None None] MFA Rules not processed for user `ba8c7348ad2e457e80e1fdd2d93aacc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:54.102422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4115d632-e38e-4bb3-b481-5dce2c5c90b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:54.194870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:54.273730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 866/1738] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:54.277611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 873/1739] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:18:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:18:54.287921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e2c3a-6e54-4551-92c7-6445f1bcf2bc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:54.290494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a92e2c3a-6e54-4551-92c7-6445f1bcf2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:54.292335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115d632-e38e-4bb3-b481-5dce2c5c90b7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:54.294385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4115d632-e38e-4bb3-b481-5dce2c5c90b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:54.307218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115d632-e38e-4bb3-b481-5dce2c5c90b7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:54.312244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:54.313159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:54.314698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:54.314698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:54.325079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:54.327394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:54.356605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 874/1740] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:54.451919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e2c3a-6e54-4551-92c7-6445f1bcf2bc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:54.467034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02acdd6-20aa-49cc-bc12-9ae53dc5788c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:54.467724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02acdd6-20aa-49cc-bc12-9ae53dc5788c service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:54.467998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02acdd6-20aa-49cc-bc12-9ae53dc5788c service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:54.468269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02acdd6-20aa-49cc-bc12-9ae53dc5788c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:54.489650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02acdd6-20aa-49cc-bc12-9ae53dc5788c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:54.491149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02acdd6-20aa-49cc-bc12-9ae53dc5788c service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:54.530580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 867/1741] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:54.631891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:54.633823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:54.660417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e2204-3c30-47d1-8575-a7cc9b9b56c4 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:54.681476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:54.692971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:54.692971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:54.692971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:54.754246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb9abf-9021-46a3-ba24-f01030e42fd2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:54.754521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb9abf-9021-46a3-ba24-f01030e42fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:54.754791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb9abf-9021-46a3-ba24-f01030e42fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:54.786741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-13cb9abf-9021-46a3-ba24-f01030e42fd2 None None] MFA Rules not processed for user `4c3e5c1776b8470a9b677bac8dfbc9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:54.788859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-13cb9abf-9021-46a3-ba24-f01030e42fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:54.879049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:54.881160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:54.885764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 868/1742] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:54.903891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 875/1743] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:55.054162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc883-3c47-42d9-b9c0-19d6a32ba4e5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:55.054690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc883-3c47-42d9-b9c0-19d6a32ba4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:55.054973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc883-3c47-42d9-b9c0-19d6a32ba4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:55.174765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-66fcc883-3c47-42d9-b9c0-19d6a32ba4e5 None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:55.176848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66fcc883-3c47-42d9-b9c0-19d6a32ba4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:55.386291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:55.438725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 869/1744] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:18:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:55.752811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:55.757704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:56.020823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:56.056255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf808cfd-6b87-44a7-9916-83f9efa5fc2c service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:56.065431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82e2d2-b891-46d5-a5ba-5cfdbee228e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:56.066271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e2d2-b891-46d5-a5ba-5cfdbee228e2 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:56.066680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e2d2-b891-46d5-a5ba-5cfdbee228e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:56.067440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e2d2-b891-46d5-a5ba-5cfdbee228e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:56.082375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82e2d2-b891-46d5-a5ba-5cfdbee228e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:56.084796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82e2d2-b891-46d5-a5ba-5cfdbee228e2 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:56.128289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 876/1745] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:18:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:18:56.828865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc883-3c47-42d9-b9c0-19d6a32ba4e5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:56.830521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66fcc883-3c47-42d9-b9c0-19d6a32ba4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:57.116529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc883-3c47-42d9-b9c0-19d6a32ba4e5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:57.133009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:57.133873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:57.134507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:57.134956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] PATH_INFO: `/v3/users/7f4b08ec590749dcb8618b09a4c6c969` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:57.137475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4b08ec590749dcb8618b09a4c6c969)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:57.142195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:57.265412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4b08ec590749dcb8618b09a4c6c969'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:57.267814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4b08ec590749dcb8618b09a4c6c969'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:57.325053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4b08ec590749dcb8618b09a4c6c969'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:57.339655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4b08ec590749dcb8618b09a4c6c969'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:57.356522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4b08ec590749dcb8618b09a4c6c969'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:57.371944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 870/1746] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:18:56 2022] DELETE /identity/v3/users/7f4b08ec590749dcb8618b09a4c6c969 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:57.376845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f6f6f-d381-4417-8eda-1df5ab084f98 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:57.377341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f6f6f-d381-4417-8eda-1df5ab084f98 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:57.377428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f6f6f-d381-4417-8eda-1df5ab084f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:57.456272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-478f6f6f-d381-4417-8eda-1df5ab084f98 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:18:57.462269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-478f6f6f-d381-4417-8eda-1df5ab084f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:57.720712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 877/1747] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:18:57.734466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:57.736454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:57.975148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48e186-0ed7-43c9-9d2f-22311791b222 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:57.987492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc27e-5e32-4cdc-9aee-940e50ff4f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:57.988310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc27e-5e32-4cdc-9aee-940e50ff4f06 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:57.988656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc27e-5e32-4cdc-9aee-940e50ff4f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:57.989119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc27e-5e32-4cdc-9aee-940e50ff4f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:58.014206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dc27e-5e32-4cdc-9aee-940e50ff4f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:58.018585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dc27e-5e32-4cdc-9aee-940e50ff4f06 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:58.045411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 871/1748] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:18:58.670140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f6f6f-d381-4417-8eda-1df5ab084f98 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:58.672689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-478f6f6f-d381-4417-8eda-1df5ab084f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:58.698468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f6f6f-d381-4417-8eda-1df5ab084f98 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:58.705543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:58.706474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:58.706902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:58.707369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] PATH_INFO: `/v3/projects/880fc0b619f64346903ff359716ba22a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:58.708711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=880fc0b619f64346903ff359716ba22a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:58.711174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:58.799393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880fc0b619f64346903ff359716ba22a', 'request_id': 'req-12bb5046-1882-4ac2-be44-07a151106e2c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:58.819517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880fc0b619f64346903ff359716ba22a', 'request_id': 'req-12bb5046-1882-4ac2-be44-07a151106e2c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:58.829180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880fc0b619f64346903ff359716ba22a', 'request_id': 'req-12bb5046-1882-4ac2-be44-07a151106e2c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:58.841027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] The token cache is being invalidate because project 880fc0b619f64346903ff359716ba22a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:18:58.841554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:58.842339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:58.844179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 878/1749] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:18:58 2022] DELETE /identity/v3/projects/880fc0b619f64346903ff359716ba22a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:18:58.849006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc27e-5e32-4cdc-9aee-940e50ff4f06 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:58.860223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc27e-5e32-4cdc-9aee-940e50ff4f06 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:58.874362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:58.931270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:58.931921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:58.932203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:58.932482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] PATH_INFO: `/v3/users/cc2e08815035468f978939f8494711e5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:58.933721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2e08815035468f978939f8494711e5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:58.935722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:59.017764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2e08815035468f978939f8494711e5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.020795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2e08815035468f978939f8494711e5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.039906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2e08815035468f978939f8494711e5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.048264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2e08815035468f978939f8494711e5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.058926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e2865b-a9a8-4081-beb0-3c17f2ce891f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2e08815035468f978939f8494711e5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.071738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 872/1750] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:18:58 2022] DELETE /identity/v3/users/cc2e08815035468f978939f8494711e5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:18:59.504926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:18:59.506542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:18:59.573538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb5046-1882-4ac2-be44-07a151106e2c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:18:59.704801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:18:59.706402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:59.706936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:59.707530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] PATH_INFO: `/v3/projects/8230e51deebf4ca8bd11ec8ef5e860b3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:18:59.709657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8230e51deebf4ca8bd11ec8ef5e860b3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:18:59.715283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:18:59.720484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:18:59.838009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8230e51deebf4ca8bd11ec8ef5e860b3', 'request_id': 'req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.885007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8230e51deebf4ca8bd11ec8ef5e860b3', 'request_id': 'req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.905031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8230e51deebf4ca8bd11ec8ef5e860b3', 'request_id': 'req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.960162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] The token cache is being invalidate because project 8230e51deebf4ca8bd11ec8ef5e860b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:18:59.960572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.961532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:18:59.967061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 879/1751] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:18:59 2022] DELETE /identity/v3/projects/8230e51deebf4ca8bd11ec8ef5e860b3 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:18:59.972541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd4a2b-de3e-43b1-85c8-bb23aa37ffb1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:18:59.973039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd4a2b-de3e-43b1-85c8-bb23aa37ffb1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:18:59.973455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd4a2b-de3e-43b1-85c8-bb23aa37ffb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.001376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:00.005466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:00.069868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8edd4a2b-de3e-43b1-85c8-bb23aa37ffb1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:00.077331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8edd4a2b-de3e-43b1-85c8-bb23aa37ffb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:00.222930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c2dc5-6f5a-4375-b697-e7c0657b2d04 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:00.229489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:00.230302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.230771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.231139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] PATH_INFO: `/v3/users/32e37b53d5844aa48e812855cef6c77f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.232623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e37b53d5844aa48e812855cef6c77f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:00.235185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:00.254825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 873/1752] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:00.264751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05d42a-0220-4e59-a258-9d62b3f8dedc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.266171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05d42a-0220-4e59-a258-9d62b3f8dedc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.266709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05d42a-0220-4e59-a258-9d62b3f8dedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.292646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bd05d42a-0220-4e59-a258-9d62b3f8dedc None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:00.294364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bd05d42a-0220-4e59-a258-9d62b3f8dedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:00.308729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 874/1753] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:00.309697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05d42a-0220-4e59-a258-9d62b3f8dedc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:00.321371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05d42a-0220-4e59-a258-9d62b3f8dedc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:00.326601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:00.327289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.327575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.327850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.328606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:00.328812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:00.331271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:00.366885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 875/1754] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:00.368938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:00.389699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e37b53d5844aa48e812855cef6c77f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:00.390392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e37b53d5844aa48e812855cef6c77f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:00.391104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889501c2-bfc1-44e9-b49b-0c16d8eda471 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:00.397977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:00.398669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.398951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.399221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.399992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:00.402681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e37b53d5844aa48e812855cef6c77f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:00.403390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e37b53d5844aa48e812855cef6c77f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:00.405681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:00.431811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d7da20bb-9d0e-4f9a-9f85-4b924c2dc208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e37b53d5844aa48e812855cef6c77f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:00.441368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:00.464848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 880/1755] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:18:59 2022] DELETE /identity/v3/users/32e37b53d5844aa48e812855cef6c77f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:00.469409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08147-3962-4dbe-9c62-4faf876e52f2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.470469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08147-3962-4dbe-9c62-4faf876e52f2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.470757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08147-3962-4dbe-9c62-4faf876e52f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.471021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 876/1756] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:00 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:00.472475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:00.474332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:00.526163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5bd08147-3962-4dbe-9c62-4faf876e52f2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:00.527893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5bd08147-3962-4dbe-9c62-4faf876e52f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:00.533129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:00.622467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 881/1757] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:00.631108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f45b9e-5914-4284-bfd1-968b93c7a3bd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:00.645467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb65821-8853-4759-824f-b1efce7ae515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:00.646244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb65821-8853-4759-824f-b1efce7ae515 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.647367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb65821-8853-4759-824f-b1efce7ae515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.647734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb65821-8853-4759-824f-b1efce7ae515 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.648565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb65821-8853-4759-824f-b1efce7ae515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:00.650204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08147-3962-4dbe-9c62-4faf876e52f2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:00.650856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb65821-8853-4759-824f-b1efce7ae515 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:00.651990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5bd08147-3962-4dbe-9c62-4faf876e52f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:00.716102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 877/1758] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:00 2022] POST /identity/v3/users => generated 294 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:00.720991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb65821-8853-4759-824f-b1efce7ae515 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:00.741499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb65821-8853-4759-824f-b1efce7ae515 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:00.747578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08147-3962-4dbe-9c62-4faf876e52f2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:00.752795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d7a4c-ba94-4f25-b5a8-8100100542a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:00.753710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7a4c-ba94-4f25-b5a8-8100100542a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.754008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7a4c-ba94-4f25-b5a8-8100100542a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.754324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7a4c-ba94-4f25-b5a8-8100100542a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.754741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231f740-20d2-470f-b467-35bf2ef44651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:00.755520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f740-20d2-470f-b467-35bf2ef44651 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.755881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f740-20d2-470f-b467-35bf2ef44651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.756250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f740-20d2-470f-b467-35bf2ef44651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.756996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231f740-20d2-470f-b467-35bf2ef44651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:00.761444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231f740-20d2-470f-b467-35bf2ef44651 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:00.771109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d7a4c-ba94-4f25-b5a8-8100100542a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:00.772841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d7a4c-ba94-4f25-b5a8-8100100542a1 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:00.786257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 878/1759] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:00.790844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231f740-20d2-470f-b467-35bf2ef44651 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:00.795321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 882/1760] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:00.802795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231f740-20d2-470f-b467-35bf2ef44651 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:00.815547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f659b-793c-4997-be39-c2cfcfa07b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:00.816284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f659b-793c-4997-be39-c2cfcfa07b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.816570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f659b-793c-4997-be39-c2cfcfa07b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.816838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f659b-793c-4997-be39-c2cfcfa07b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.817492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f659b-793c-4997-be39-c2cfcfa07b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:00.820804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f659b-793c-4997-be39-c2cfcfa07b1c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:00.848700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 879/1761] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:00.850224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d7a4c-ba94-4f25-b5a8-8100100542a1 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:00.873839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d7a4c-ba94-4f25-b5a8-8100100542a1 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:00.883866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b699eb1-466b-4661-9810-7735c87a318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:00.884556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b699eb1-466b-4661-9810-7735c87a318c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.884835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b699eb1-466b-4661-9810-7735c87a318c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.885130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b699eb1-466b-4661-9810-7735c87a318c admin admin] PATH_INFO: `/v3/projects/c020af9ba5cc41aa878027184cd419ae/users/b5277af7465e4b87ae4ce17e1010a98b/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:00.917948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b699eb1-466b-4661-9810-7735c87a318c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c020af9ba5cc41aa878027184cd419ae, user_id=b5277af7465e4b87ae4ce17e1010a98b, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:00.933747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b699eb1-466b-4661-9810-7735c87a318c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:00.973523 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 883/1762] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:00 2022] PUT /identity/v3/projects/c020af9ba5cc41aa878027184cd419ae/users/b5277af7465e4b87ae4ce17e1010a98b/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:00.989356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1e88-139c-4c9f-bbae-4b29dc734353 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:00.989640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1e88-139c-4c9f-bbae-4b29dc734353 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:00.990167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1e88-139c-4c9f-bbae-4b29dc734353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:01.058734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2fed1e88-139c-4c9f-bbae-4b29dc734353 None None] MFA Rules not processed for user `b5277af7465e4b87ae4ce17e1010a98b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:01.060306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2fed1e88-139c-4c9f-bbae-4b29dc734353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:01.284335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b699eb1-466b-4661-9810-7735c87a318c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:01.291977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 880/1763] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:01.438921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b699eb1-466b-4661-9810-7735c87a318c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:01.448967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:01.449934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:01.450210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:01.450487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:01.456429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:01.458674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:01.523614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 884/1764] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:01 2022] POST /identity/v3/projects => generated 387 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:01.532387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1e88-139c-4c9f-bbae-4b29dc734353 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:01.552176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:01.553825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1e88-139c-4c9f-bbae-4b29dc734353 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:01.556560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:01.612734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:01.742335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:01.775957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52ffcc-9e88-4994-bb67-774b2edf1d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:01.776606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ffcc-9e88-4994-bb67-774b2edf1d25 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:01.776891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ffcc-9e88-4994-bb67-774b2edf1d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:01.777164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ffcc-9e88-4994-bb67-774b2edf1d25 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:01.777812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52ffcc-9e88-4994-bb67-774b2edf1d25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:01.779876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52ffcc-9e88-4994-bb67-774b2edf1d25 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:01.794872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0042051-89ec-4745-83d0-85a8f09a352a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:01.802457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:01.803535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:01.803832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:01.804160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] PATH_INFO: `/v3/projects/4ef29c50c368400780306d7344d0f15a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:01.805642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef29c50c368400780306d7344d0f15a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:01.808013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:01.817645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 881/1765] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:01 2022] POST /identity/v3/users => generated 294 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:01.822242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52ffcc-9e88-4994-bb67-774b2edf1d25 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:01.845492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52ffcc-9e88-4994-bb67-774b2edf1d25 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:01.854111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cc526-806d-4805-8a5c-cbeaa95a7dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:01.855151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cc526-806d-4805-8a5c-cbeaa95a7dad admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:01.855794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cc526-806d-4805-8a5c-cbeaa95a7dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:01.858389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cc526-806d-4805-8a5c-cbeaa95a7dad admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:01.859291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cc526-806d-4805-8a5c-cbeaa95a7dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:01.863989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cc526-806d-4805-8a5c-cbeaa95a7dad admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:01.910123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 882/1766] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:01.912341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cc526-806d-4805-8a5c-cbeaa95a7dad admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:01.915726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef29c50c368400780306d7344d0f15a', 'request_id': 'req-7a78e6d2-da0d-4923-82c8-d934fc1c108e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:01.924126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef29c50c368400780306d7344d0f15a', 'request_id': 'req-7a78e6d2-da0d-4923-82c8-d934fc1c108e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:01.927170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cc526-806d-4805-8a5c-cbeaa95a7dad admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:01.941889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef29c50c368400780306d7344d0f15a', 'request_id': 'req-7a78e6d2-da0d-4923-82c8-d934fc1c108e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:01.955313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] The token cache is being invalidate because project 4ef29c50c368400780306d7344d0f15a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:01.955614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:01.956421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:01.962404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 885/1767] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:01 2022] DELETE /identity/v3/projects/4ef29c50c368400780306d7344d0f15a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:01.967495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:01.973033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:02.014710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:02.015374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.015649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.015911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.016662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:02.020013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:02.039516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 883/1768] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:02.051225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:02.056245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:02.077938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a38d6-d162-4922-b692-3b8d26709380 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:02.085935 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f6d45-ab1d-4519-a0bc-2fbc8b94d79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:02.087206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f6d45-ab1d-4519-a0bc-2fbc8b94d79c admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.087856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f6d45-ab1d-4519-a0bc-2fbc8b94d79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.088344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f6d45-ab1d-4519-a0bc-2fbc8b94d79c admin admin] PATH_INFO: `/v3/projects/d3a18c25c7634825ac2365b48dd5d208/users/41e3cec3a49d484d81a0aca4758635c4/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.113652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78e6d2-da0d-4923-82c8-d934fc1c108e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:02.129246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f6d45-ab1d-4519-a0bc-2fbc8b94d79c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a18c25c7634825ac2365b48dd5d208, user_id=41e3cec3a49d484d81a0aca4758635c4, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:02.129393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:02.130160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.130826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.131196 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] PATH_INFO: `/v3/users/d84c43e952924fe5bd3392fd2d14aeb6` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.132526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f6d45-ab1d-4519-a0bc-2fbc8b94d79c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:02.134120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84c43e952924fe5bd3392fd2d14aeb6)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:02.136914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:02.154490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 884/1769] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:02 2022] PUT /identity/v3/projects/d3a18c25c7634825ac2365b48dd5d208/users/41e3cec3a49d484d81a0aca4758635c4/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:02.168561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1ef6-75ec-4716-b09b-7727eb1e9660 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.168959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1ef6-75ec-4716-b09b-7727eb1e9660 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.169260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1ef6-75ec-4716-b09b-7727eb1e9660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.230229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:02.231620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f8da1ef6-75ec-4716-b09b-7727eb1e9660 None None] MFA Rules not processed for user `41e3cec3a49d484d81a0aca4758635c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:02.232994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f8da1ef6-75ec-4716-b09b-7727eb1e9660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:02.239110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84c43e952924fe5bd3392fd2d14aeb6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:02.240609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84c43e952924fe5bd3392fd2d14aeb6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:02.251890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84c43e952924fe5bd3392fd2d14aeb6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:02.252770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84c43e952924fe5bd3392fd2d14aeb6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:02.275439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84c43e952924fe5bd3392fd2d14aeb6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:02.289659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 886/1770] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:01 2022] DELETE /identity/v3/users/d84c43e952924fe5bd3392fd2d14aeb6 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:02.342670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 885/1771] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:02.344745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:02.346469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:02.443466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a63b7-ae0f-4d80-b9ed-591072a7cae4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:02.456543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091bbca-7e82-4610-8fda-a8e1054a784b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:02.457186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091bbca-7e82-4610-8fda-a8e1054a784b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.457445 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091bbca-7e82-4610-8fda-a8e1054a784b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.457732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091bbca-7e82-4610-8fda-a8e1054a784b admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.458396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091bbca-7e82-4610-8fda-a8e1054a784b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:02.465397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091bbca-7e82-4610-8fda-a8e1054a784b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:02.500454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 887/1772] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:02 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:02.501502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da1ef6-75ec-4716-b09b-7727eb1e9660 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:02.583427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:02.629092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da1ef6-75ec-4716-b09b-7727eb1e9660 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:02.709429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:02.710227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.710600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.710967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.711776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:02.714002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:02.754985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 886/1773] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:02 2022] POST /identity/v3/users => generated 299 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:02.760090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091bbca-7e82-4610-8fda-a8e1054a784b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:02.778598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091bbca-7e82-4610-8fda-a8e1054a784b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:02.792165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d5718-9d38-4bf8-9606-c2ce99ff92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:02.792815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5718-9d38-4bf8-9606-c2ce99ff92ae admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.793101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5718-9d38-4bf8-9606-c2ce99ff92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.793427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5718-9d38-4bf8-9606-c2ce99ff92ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.794059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5718-9d38-4bf8-9606-c2ce99ff92ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:02.809964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5718-9d38-4bf8-9606-c2ce99ff92ae admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:02.815116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:02.817153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:02.844528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a910be5-82a2-4940-82c1-a94c1e704324 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:02.848488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 888/1774] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:02.854104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d5718-9d38-4bf8-9606-c2ce99ff92ae admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:02.855079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:02.857582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.858793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.859130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] PATH_INFO: `/v3/projects/d935e3ec3875494abe1c5cd68ad5084b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.861839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d935e3ec3875494abe1c5cd68ad5084b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:02.877766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:02.882616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d5718-9d38-4bf8-9606-c2ce99ff92ae admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:02.900299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192939c0-a899-4fe3-8941-df6f332fa412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:02.902137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192939c0-a899-4fe3-8941-df6f332fa412 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:02.905930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192939c0-a899-4fe3-8941-df6f332fa412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:02.906521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192939c0-a899-4fe3-8941-df6f332fa412 admin admin] PATH_INFO: `/v3/projects/ede043b2d6994d10b3d6111fc8bf4686/users/89f6116f611d4b49bdf66dd74d7d7d8e/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:02.959921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192939c0-a899-4fe3-8941-df6f332fa412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede043b2d6994d10b3d6111fc8bf4686, user_id=89f6116f611d4b49bdf66dd74d7d7d8e, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:02.962689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192939c0-a899-4fe3-8941-df6f332fa412 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:02.982890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 889/1775] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:02 2022] PUT /identity/v3/projects/ede043b2d6994d10b3d6111fc8bf4686/users/89f6116f611d4b49bdf66dd74d7d7d8e/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:02.987492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192939c0-a899-4fe3-8941-df6f332fa412 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.017975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd935e3ec3875494abe1c5cd68ad5084b', 'request_id': 'req-be8971e7-206b-4162-98a8-bc832785bf85'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:03.026281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd935e3ec3875494abe1c5cd68ad5084b', 'request_id': 'req-be8971e7-206b-4162-98a8-bc832785bf85'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:03.035607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd935e3ec3875494abe1c5cd68ad5084b', 'request_id': 'req-be8971e7-206b-4162-98a8-bc832785bf85'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:03.058584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] The token cache is being invalidate because project d935e3ec3875494abe1c5cd68ad5084b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:03.058940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:03.059723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-be8971e7-206b-4162-98a8-bc832785bf85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:03.062490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 887/1776] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:02 2022] DELETE /identity/v3/projects/d935e3ec3875494abe1c5cd68ad5084b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:03.068674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192939c0-a899-4fe3-8941-df6f332fa412 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.068886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665268-166c-44de-830d-efad020e9819 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.069173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665268-166c-44de-830d-efad020e9819 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.069447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665268-166c-44de-830d-efad020e9819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.095503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-30665268-166c-44de-830d-efad020e9819 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:03.096808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-30665268-166c-44de-830d-efad020e9819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:03.100765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:03.176479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:03.202583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 888/1777] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:03.202870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.202870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.202870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.202870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.203415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:03.203734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f729a-e00b-4b11-a594-0adadab0172f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.203917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:03.204653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f729a-e00b-4b11-a594-0adadab0172f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.205236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f729a-e00b-4b11-a594-0adadab0172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.243302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 890/1778] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:03.248289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.256714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:03.257103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3e9f729a-e00b-4b11-a594-0adadab0172f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:03.257103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3e9f729a-e00b-4b11-a594-0adadab0172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:03.271507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 889/1779] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:03.271752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acb3e9-c764-4d25-87a1-389c3332b119 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.272960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f729a-e00b-4b11-a594-0adadab0172f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.294669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f729a-e00b-4b11-a594-0adadab0172f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.295195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ba1ad-b0ca-4ac0-8966-b269bc85b5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.295919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba1ad-b0ca-4ac0-8966-b269bc85b5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.296195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba1ad-b0ca-4ac0-8966-b269bc85b5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.296456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba1ad-b0ca-4ac0-8966-b269bc85b5b2 admin admin] PATH_INFO: `/v3/projects/ede043b2d6994d10b3d6111fc8bf4686/users/89f6116f611d4b49bdf66dd74d7d7d8e/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.304114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba1ad-b0ca-4ac0-8966-b269bc85b5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede043b2d6994d10b3d6111fc8bf4686, user_id=89f6116f611d4b49bdf66dd74d7d7d8e, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:03.306894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba1ad-b0ca-4ac0-8966-b269bc85b5b2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:03.340092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.340814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.341100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.341370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.342073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:03.342277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 891/1780] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:03 2022] PUT /identity/v3/projects/ede043b2d6994d10b3d6111fc8bf4686/users/89f6116f611d4b49bdf66dd74d7d7d8e/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:03.342386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:03.344516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:03.363184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f5f97-5368-4325-8994-48f273bbaef7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.363887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f5f97-5368-4325-8994-48f273bbaef7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.364405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f5f97-5368-4325-8994-48f273bbaef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.379786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 890/1781] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:03.380905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.402570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baf102-39db-4ae4-af59-d07a76385fcc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.409684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ab1f5f97-5368-4325-8994-48f273bbaef7 None None] MFA Rules not processed for user `89f6116f611d4b49bdf66dd74d7d7d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:03.410991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ab1f5f97-5368-4325-8994-48f273bbaef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:03.498776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.499715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.500482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.501071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.501998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:03.504286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:03.544069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 892/1782] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:03.546095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 891/1783] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:03 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:03.552553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f5f97-5368-4325-8994-48f273bbaef7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.575899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f5f97-5368-4325-8994-48f273bbaef7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.669523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.671575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:03.723091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef1980-d982-46b4-9b9e-2375d385c9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.723861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef1980-d982-46b4-9b9e-2375d385c9fb admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.724200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef1980-d982-46b4-9b9e-2375d385c9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.724530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef1980-d982-46b4-9b9e-2375d385c9fb admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.725268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef1980-d982-46b4-9b9e-2375d385c9fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:03.742682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef1980-d982-46b4-9b9e-2375d385c9fb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:03.822300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 893/1784] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:03 2022] POST /identity/v3/users => generated 294 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:03.827538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef1980-d982-46b4-9b9e-2375d385c9fb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.847007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef1980-d982-46b4-9b9e-2375d385c9fb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.856294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70669df9-b58a-47bc-bd10-0a67f70a02bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.857850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70669df9-b58a-47bc-bd10-0a67f70a02bb admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.858329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70669df9-b58a-47bc-bd10-0a67f70a02bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.858782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70669df9-b58a-47bc-bd10-0a67f70a02bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.859853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70669df9-b58a-47bc-bd10-0a67f70a02bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:03.862254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70669df9-b58a-47bc-bd10-0a67f70a02bb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:03.868899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb504334-8003-4454-90c8-e045cb8cff5d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.886882 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 894/1785] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:03.889011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70669df9-b58a-47bc-bd10-0a67f70a02bb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.892616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc5fe4-ff7d-47a6-9f99-21b870452ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.898432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc5fe4-ff7d-47a6-9f99-21b870452ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.898945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc5fe4-ff7d-47a6-9f99-21b870452ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.900131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc5fe4-ff7d-47a6-9f99-21b870452ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.905127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70669df9-b58a-47bc-bd10-0a67f70a02bb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.913490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a4473-3d28-4ab8-8207-e5a98297b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.914191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4473-3d28-4ab8-8207-e5a98297b0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.916093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4473-3d28-4ab8-8207-e5a98297b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.917345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4473-3d28-4ab8-8207-e5a98297b0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:03.918181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a4473-3d28-4ab8-8207-e5a98297b0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:03.920125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a4473-3d28-4ab8-8207-e5a98297b0b7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:03.924661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:03.950503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 895/1786] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:03.952785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a4473-3d28-4ab8-8207-e5a98297b0b7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:03.970539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a4473-3d28-4ab8-8207-e5a98297b0b7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:03.976131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222ae1d-dfd3-422c-9e2c-3c0126fd0a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:03.977081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222ae1d-dfd3-422c-9e2c-3c0126fd0a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:03.977424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222ae1d-dfd3-422c-9e2c-3c0126fd0a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:03.977769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222ae1d-dfd3-422c-9e2c-3c0126fd0a73 admin admin] PATH_INFO: `/v3/projects/59322390bee6442089a11c06c5ddbbe9/users/aed8450a57b04090b638c9dbef51d612/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:04.007870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222ae1d-dfd3-422c-9e2c-3c0126fd0a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59322390bee6442089a11c06c5ddbbe9, user_id=aed8450a57b04090b638c9dbef51d612, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:04.010729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222ae1d-dfd3-422c-9e2c-3c0126fd0a73 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:04.015178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc5fe4-ff7d-47a6-9f99-21b870452ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:04.018310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc5fe4-ff7d-47a6-9f99-21b870452ab1 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:04.030781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 892/1787] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:04.037154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 896/1788] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:03 2022] PUT /identity/v3/projects/59322390bee6442089a11c06c5ddbbe9/users/aed8450a57b04090b638c9dbef51d612/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:04.049689 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7ef5-ea8c-4985-9dc5-18836e1ce52f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:04.049972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7ef5-ea8c-4985-9dc5-18836e1ce52f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:04.050598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7ef5-ea8c-4985-9dc5-18836e1ce52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:04.082756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:04.099104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-09eb7ef5-ea8c-4985-9dc5-18836e1ce52f None None] MFA Rules not processed for user `aed8450a57b04090b638c9dbef51d612`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:04.101645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-09eb7ef5-ea8c-4985-9dc5-18836e1ce52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:04.255395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 893/1789] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:04.259016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222ae1d-dfd3-422c-9e2c-3c0126fd0a73 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:04.412727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222ae1d-dfd3-422c-9e2c-3c0126fd0a73 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:04.419869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554afa3-2e72-4bf7-8230-2a46c3b5e76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:04.420798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554afa3-2e72-4bf7-8230-2a46c3b5e76f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:04.421088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554afa3-2e72-4bf7-8230-2a46c3b5e76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:04.421617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554afa3-2e72-4bf7-8230-2a46c3b5e76f admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:04.422380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554afa3-2e72-4bf7-8230-2a46c3b5e76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:04.425587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554afa3-2e72-4bf7-8230-2a46c3b5e76f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:04.489128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 897/1790] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:04 2022] POST /identity/v3/projects => generated 387 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:04.494232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb7ef5-ea8c-4985-9dc5-18836e1ce52f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:04.507818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb7ef5-ea8c-4985-9dc5-18836e1ce52f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:04.591656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c695db2-8194-4e6e-b2d8-88e0adb7be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:04.592362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c695db2-8194-4e6e-b2d8-88e0adb7be75 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:04.592801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c695db2-8194-4e6e-b2d8-88e0adb7be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:04.593154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c695db2-8194-4e6e-b2d8-88e0adb7be75 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:04.593813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c695db2-8194-4e6e-b2d8-88e0adb7be75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:04.596181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c695db2-8194-4e6e-b2d8-88e0adb7be75 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:04.633379 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 894/1791] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:04 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:04.638288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554afa3-2e72-4bf7-8230-2a46c3b5e76f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:04.658769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554afa3-2e72-4bf7-8230-2a46c3b5e76f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:04.666436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8113ab-cc47-4af9-9111-454dc5a4ab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:04.667550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8113ab-cc47-4af9-9111-454dc5a4ab79 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:04.667550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8113ab-cc47-4af9-9111-454dc5a4ab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:04.667774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8113ab-cc47-4af9-9111-454dc5a4ab79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:04.668733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8113ab-cc47-4af9-9111-454dc5a4ab79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:04.671444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8113ab-cc47-4af9-9111-454dc5a4ab79 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:04.692696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 898/1792] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:04.696829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c695db2-8194-4e6e-b2d8-88e0adb7be75 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:04.708938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c695db2-8194-4e6e-b2d8-88e0adb7be75 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:04.714563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4d0b4-c513-4e27-8e11-b2b3f0b642a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:04.715118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4d0b4-c513-4e27-8e11-b2b3f0b642a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:04.715489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4d0b4-c513-4e27-8e11-b2b3f0b642a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:04.715704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4d0b4-c513-4e27-8e11-b2b3f0b642a3 admin admin] PATH_INFO: `/v3/projects/5efa5419cf7c4a6885398543491660fa/users/d1743fa7d1dd4df9aee9182d27a8c9a0/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:04.736456 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4d0b4-c513-4e27-8e11-b2b3f0b642a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efa5419cf7c4a6885398543491660fa, user_id=d1743fa7d1dd4df9aee9182d27a8c9a0, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:04.739452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4d0b4-c513-4e27-8e11-b2b3f0b642a3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:04.789578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 895/1793] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:04 2022] PUT /identity/v3/projects/5efa5419cf7c4a6885398543491660fa/users/d1743fa7d1dd4df9aee9182d27a8c9a0/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:04.793996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8113ab-cc47-4af9-9111-454dc5a4ab79 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:04.811025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8113ab-cc47-4af9-9111-454dc5a4ab79 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:05.127161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7635aa5-c4b8-413d-8747-dc21c8cb3b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:05.129635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635aa5-c4b8-413d-8747-dc21c8cb3b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:05.133650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635aa5-c4b8-413d-8747-dc21c8cb3b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:05.137745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635aa5-c4b8-413d-8747-dc21c8cb3b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:05.137745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7635aa5-c4b8-413d-8747-dc21c8cb3b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:05.152565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7635aa5-c4b8-413d-8747-dc21c8cb3b36 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:05.240039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4d0b4-c513-4e27-8e11-b2b3f0b642a3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:05.242849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 899/1794] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:05.279760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4d0b4-c513-4e27-8e11-b2b3f0b642a3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:05.300648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff024-16a3-403a-bb6b-6874dac1b363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:05.303423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff024-16a3-403a-bb6b-6874dac1b363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:05.304517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff024-16a3-403a-bb6b-6874dac1b363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:05.304789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff024-16a3-403a-bb6b-6874dac1b363 admin admin] PATH_INFO: `/v3/projects/5efa5419cf7c4a6885398543491660fa/users/d1743fa7d1dd4df9aee9182d27a8c9a0/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:05.308545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ff024-16a3-403a-bb6b-6874dac1b363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efa5419cf7c4a6885398543491660fa, user_id=d1743fa7d1dd4df9aee9182d27a8c9a0, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:05.311745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ff024-16a3-403a-bb6b-6874dac1b363 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:05.328220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 896/1795] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:05 2022] PUT /identity/v3/projects/5efa5419cf7c4a6885398543491660fa/users/d1743fa7d1dd4df9aee9182d27a8c9a0/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:05.350736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910a068-6d8d-4870-8b18-c4b830694700 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:05.351485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910a068-6d8d-4870-8b18-c4b830694700 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:05.352150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910a068-6d8d-4870-8b18-c4b830694700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:05.354220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff024-16a3-403a-bb6b-6874dac1b363 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:05.538747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5910a068-6d8d-4870-8b18-c4b830694700 None None] MFA Rules not processed for user `d1743fa7d1dd4df9aee9182d27a8c9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:05.538747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5910a068-6d8d-4870-8b18-c4b830694700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:05.672884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff024-16a3-403a-bb6b-6874dac1b363 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:05.694773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:05.696828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:05.697150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:05.697428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:05.709476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:05.719021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:05.741725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:05.914469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:05.918393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:05.937823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 900/1796] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 590 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:05.939649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a2295-d264-4bd2-9443-45fbb9f9569d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:05.939754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a2295-d264-4bd2-9443-45fbb9f9569d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:05.940482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a2295-d264-4bd2-9443-45fbb9f9569d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:05.947926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 897/1797] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:06.035661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:06.037366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:06.049206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4f5a2295-d264-4bd2-9443-45fbb9f9569d None None] MFA Rules not processed for user `ba8c7348ad2e457e80e1fdd2d93aacc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:06.052979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4f5a2295-d264-4bd2-9443-45fbb9f9569d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:06.056225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70cad1-a233-45b1-94a5-abae9fc96d9c service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:06.086772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fb124-7075-41a0-b64a-14887fde6376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:06.091447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fb124-7075-41a0-b64a-14887fde6376 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:06.091447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fb124-7075-41a0-b64a-14887fde6376 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:06.091447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fb124-7075-41a0-b64a-14887fde6376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:06.114523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fb124-7075-41a0-b64a-14887fde6376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:06.116801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fb124-7075-41a0-b64a-14887fde6376 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:06.135385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 898/1798] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:06.193222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 901/1799] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 31 12:19:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:06.194155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fb124-7075-41a0-b64a-14887fde6376 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:06.329125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fb124-7075-41a0-b64a-14887fde6376 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:06.338131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:06.339847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:06.339847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:06.339847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] PATH_INFO: `/v3/users/aed8450a57b04090b638c9dbef51d612` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:06.341393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed8450a57b04090b638c9dbef51d612)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:06.346750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:06.435867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed8450a57b04090b638c9dbef51d612'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:06.436995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed8450a57b04090b638c9dbef51d612'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:06.478442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed8450a57b04090b638c9dbef51d612'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:06.501281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed8450a57b04090b638c9dbef51d612'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:06.526916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed8450a57b04090b638c9dbef51d612'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:06.538130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 899/1800] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:06 2022] DELETE /identity/v3/users/aed8450a57b04090b638c9dbef51d612 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:06.547981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5adfa1-e56e-47cd-a083-61d9d2b16f3d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:06.549557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5adfa1-e56e-47cd-a083-61d9d2b16f3d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:06.551422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5adfa1-e56e-47cd-a083-61d9d2b16f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:06.580590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fd5adfa1-e56e-47cd-a083-61d9d2b16f3d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:06.581878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fd5adfa1-e56e-47cd-a083-61d9d2b16f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:06.670990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 902/1801] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:06.682551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:06.684304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:06.700016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:06.772046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a1c15-6ef5-4587-a7c9-b69a45677607 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:06.778850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc53632-df8d-495f-addb-ef33ff0189ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:06.779076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53632-df8d-495f-addb-ef33ff0189ad service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:06.779311 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53632-df8d-495f-addb-ef33ff0189ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:06.779577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53632-df8d-495f-addb-ef33ff0189ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:06.797290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc53632-df8d-495f-addb-ef33ff0189ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:06.798896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc53632-df8d-495f-addb-ef33ff0189ad service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:06.811197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 900/1802] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:07.345489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5adfa1-e56e-47cd-a083-61d9d2b16f3d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:07.346955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fd5adfa1-e56e-47cd-a083-61d9d2b16f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:07.361235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5adfa1-e56e-47cd-a083-61d9d2b16f3d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:07.368343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:07.368988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:07.369262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:07.369688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] PATH_INFO: `/v3/projects/59322390bee6442089a11c06c5ddbbe9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:07.370919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59322390bee6442089a11c06c5ddbbe9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:07.383168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:07.436493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59322390bee6442089a11c06c5ddbbe9', 'request_id': 'req-74ecc713-fef6-463a-9b60-3e0ef8824868'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.443430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59322390bee6442089a11c06c5ddbbe9', 'request_id': 'req-74ecc713-fef6-463a-9b60-3e0ef8824868'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.452367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59322390bee6442089a11c06c5ddbbe9', 'request_id': 'req-74ecc713-fef6-463a-9b60-3e0ef8824868'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.460141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:07.464113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] The token cache is being invalidate because project 59322390bee6442089a11c06c5ddbbe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:07.464406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.465311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.467550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 903/1803] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:07 2022] DELETE /identity/v3/projects/59322390bee6442089a11c06c5ddbbe9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:07.471958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc53632-df8d-495f-addb-ef33ff0189ad service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:07.481981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc53632-df8d-495f-addb-ef33ff0189ad service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:07.539432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:07.540077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:07.540352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:07.540624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] PATH_INFO: `/v3/users/d1743fa7d1dd4df9aee9182d27a8c9a0` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:07.541767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1743fa7d1dd4df9aee9182d27a8c9a0)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:07.543737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:07.617306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1743fa7d1dd4df9aee9182d27a8c9a0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.618339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1743fa7d1dd4df9aee9182d27a8c9a0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.639973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1743fa7d1dd4df9aee9182d27a8c9a0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.650480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1743fa7d1dd4df9aee9182d27a8c9a0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.663892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-52ac5db4-28ed-4435-b056-339a64c7ba6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1743fa7d1dd4df9aee9182d27a8c9a0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:07.678806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 901/1804] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:07 2022] DELETE /identity/v3/users/d1743fa7d1dd4df9aee9182d27a8c9a0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:08.030246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:08.032182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:08.056563 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecc713-fef6-463a-9b60-3e0ef8824868 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:08.160191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:08.161108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:08.161538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:08.161942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] PATH_INFO: `/v3/projects/5efa5419cf7c4a6885398543491660fa` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:08.163308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5efa5419cf7c4a6885398543491660fa)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:08.165454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:08.260511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5efa5419cf7c4a6885398543491660fa', 'request_id': 'req-80ec701b-1742-48cd-8436-dbbebd9b63d8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:08.267914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5efa5419cf7c4a6885398543491660fa', 'request_id': 'req-80ec701b-1742-48cd-8436-dbbebd9b63d8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:08.279067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5efa5419cf7c4a6885398543491660fa', 'request_id': 'req-80ec701b-1742-48cd-8436-dbbebd9b63d8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:08.326918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] The token cache is being invalidate because project 5efa5419cf7c4a6885398543491660fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:08.328986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:08.330200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80ec701b-1742-48cd-8436-dbbebd9b63d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:08.336520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 904/1805] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:08 2022] DELETE /identity/v3/projects/5efa5419cf7c4a6885398543491660fa => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:08.342310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6717-a815-47cd-a20d-87243b001821 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:08.342883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6717-a815-47cd-a20d-87243b001821 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:08.342996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6717-a815-47cd-a20d-87243b001821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:08.398061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-26ad6717-a815-47cd-a20d-87243b001821 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:08.400903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-26ad6717-a815-47cd-a20d-87243b001821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:08.465651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:08.509093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 902/1806] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:08.516262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a98228-6c61-4a6c-8527-61ba595e8969 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:08.516971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a98228-6c61-4a6c-8527-61ba595e8969 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:08.516971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a98228-6c61-4a6c-8527-61ba595e8969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:08.563902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-28a98228-6c61-4a6c-8527-61ba595e8969 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:08.565258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-28a98228-6c61-4a6c-8527-61ba595e8969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:08.577289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 905/1807] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:08.582063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad6717-a815-47cd-a20d-87243b001821 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:08.593026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad6717-a815-47cd-a20d-87243b001821 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:08.614558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:08.615288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:08.615565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:08.615842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:08.616656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:08.616959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:08.618786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:08.674691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 903/1808] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:08.676584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a98228-6c61-4a6c-8527-61ba595e8969 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:08.697086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a98228-6c61-4a6c-8527-61ba595e8969 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:08.710416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbbbb5-4d6a-49f8-9c07-1b5c02cfd5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:08.711906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbbbb5-4d6a-49f8-9c07-1b5c02cfd5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:08.712194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbbbb5-4d6a-49f8-9c07-1b5c02cfd5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:08.712478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbbbb5-4d6a-49f8-9c07-1b5c02cfd5b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:08.713162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbbbb5-4d6a-49f8-9c07-1b5c02cfd5b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:08.715285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbbbb5-4d6a-49f8-9c07-1b5c02cfd5b0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:08.776198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 906/1809] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:08 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:08.780653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:08.792453 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b1fd2-397d-41c3-8077-c9799ba52c0e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:08.913537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c94dc-f183-42ff-b81b-b8530bf56a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:08.914914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c94dc-f183-42ff-b81b-b8530bf56a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:08.915707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c94dc-f183-42ff-b81b-b8530bf56a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:08.916292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c94dc-f183-42ff-b81b-b8530bf56a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:08.917154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c94dc-f183-42ff-b81b-b8530bf56a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:08.919736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c94dc-f183-42ff-b81b-b8530bf56a4f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:08.970973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 904/1810] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:08 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:08.976743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbbbb5-4d6a-49f8-9c07-1b5c02cfd5b0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:08.988470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbbbb5-4d6a-49f8-9c07-1b5c02cfd5b0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:08.994116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406e75a-3e9b-47d5-9091-da900c78753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:08.994954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406e75a-3e9b-47d5-9091-da900c78753d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:08.995416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406e75a-3e9b-47d5-9091-da900c78753d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:08.995798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406e75a-3e9b-47d5-9091-da900c78753d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:08.996362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406e75a-3e9b-47d5-9091-da900c78753d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:08.997993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406e75a-3e9b-47d5-9091-da900c78753d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:09.021383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c94dc-f183-42ff-b81b-b8530bf56a4f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:09.023667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 907/1811] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:09.037124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c94dc-f183-42ff-b81b-b8530bf56a4f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:09.044699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcf42f-2f0b-421d-b8ac-9c281758e262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:09.045523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf42f-2f0b-421d-b8ac-9c281758e262 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:09.045947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf42f-2f0b-421d-b8ac-9c281758e262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:09.046262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf42f-2f0b-421d-b8ac-9c281758e262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:09.047289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcf42f-2f0b-421d-b8ac-9c281758e262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:09.049299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcf42f-2f0b-421d-b8ac-9c281758e262 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:09.069388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 905/1812] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:09.076460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406e75a-3e9b-47d5-9091-da900c78753d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:09.090905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406e75a-3e9b-47d5-9091-da900c78753d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:09.096250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:09.096935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:09.097198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:09.097467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] PATH_INFO: `/v3/projects/c15bed7fec3c4f81a73182567caa30bd/users/befc6c8959484a18b0a0a4f01a6bb44d/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:09.114490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15bed7fec3c4f81a73182567caa30bd, user_id=befc6c8959484a18b0a0a4f01a6bb44d, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:09.117289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:09.133512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 908/1813] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:09 2022] PUT /identity/v3/projects/c15bed7fec3c4f81a73182567caa30bd/users/befc6c8959484a18b0a0a4f01a6bb44d/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:09.139381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6eb1b-664a-4ecc-b0d7-483b6c67bb06 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:09.139794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6eb1b-664a-4ecc-b0d7-483b6c67bb06 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:09.140014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6eb1b-664a-4ecc-b0d7-483b6c67bb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:09.177243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7ce6eb1b-664a-4ecc-b0d7-483b6c67bb06 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:09.178531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7ce6eb1b-664a-4ecc-b0d7-483b6c67bb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:09.256043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 906/1814] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:09.264457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:09.267608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:09.303528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:09.332596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25ac43-f241-465a-8993-eb0f0bbb5427 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:09.337079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecc148-7e3f-4bba-90dd-bea1a298b7fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:09.337734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecc148-7e3f-4bba-90dd-bea1a298b7fb service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:09.338014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecc148-7e3f-4bba-90dd-bea1a298b7fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:09.338283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecc148-7e3f-4bba-90dd-bea1a298b7fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:09.358092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecc148-7e3f-4bba-90dd-bea1a298b7fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:09.359754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecc148-7e3f-4bba-90dd-bea1a298b7fb service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:09.375020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 909/1815] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:09.988631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6eb1b-664a-4ecc-b0d7-483b6c67bb06 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:09.991112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7ce6eb1b-664a-4ecc-b0d7-483b6c67bb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:10.031943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6eb1b-664a-4ecc-b0d7-483b6c67bb06 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:10.055295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecc148-7e3f-4bba-90dd-bea1a298b7fb service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:10.072419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecc148-7e3f-4bba-90dd-bea1a298b7fb service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:10.074513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:10.075619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:10.075902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:10.079470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:10.089015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191dcd0-f69e-4e55-86dc-a81f9ac80cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:10.089670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191dcd0-f69e-4e55-86dc-a81f9ac80cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:10.090391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191dcd0-f69e-4e55-86dc-a81f9ac80cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:10.090876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191dcd0-f69e-4e55-86dc-a81f9ac80cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:10.091816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:10.094213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d191dcd0-f69e-4e55-86dc-a81f9ac80cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:10.109489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191dcd0-f69e-4e55-86dc-a81f9ac80cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:10.111492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191dcd0-f69e-4e55-86dc-a81f9ac80cf2 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:10.134941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:10.137152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:10.147694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 907/1816] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:10.148762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:10.159267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33178f-6c90-43ad-83dc-b1db0b0befde service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:10.165054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384002f-0869-48a3-8807-92b6bee99195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:10.165709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384002f-0869-48a3-8807-92b6bee99195 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:10.165955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384002f-0869-48a3-8807-92b6bee99195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:10.166273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384002f-0869-48a3-8807-92b6bee99195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:10.181991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 910/1817] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:10.206452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384002f-0869-48a3-8807-92b6bee99195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:10.214696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384002f-0869-48a3-8807-92b6bee99195 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:10.239932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 908/1818] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:12.551536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610a01f-04b0-4711-804a-417f3fc88d73 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:12.552537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610a01f-04b0-4711-804a-417f3fc88d73 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:12.552835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610a01f-04b0-4711-804a-417f3fc88d73 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:12.554929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 911/1819] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:19:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:19:12.573843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b63fd-ad92-4e6e-9f23-25a4df205e71 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:12.574404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b63fd-ad92-4e6e-9f23-25a4df205e71 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:12.574404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b63fd-ad92-4e6e-9f23-25a4df205e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:12.618184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e161-c5ac-45b3-8f9f-af1eb01fafb0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:12.618467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e161-c5ac-45b3-8f9f-af1eb01fafb0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:12.619049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e161-c5ac-45b3-8f9f-af1eb01fafb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:12.620279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-345b63fd-ad92-4e6e-9f23-25a4df205e71 None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:12.621786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-345b63fd-ad92-4e6e-9f23-25a4df205e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:12.653879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:12.684765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0194e161-c5ac-45b3-8f9f-af1eb01fafb0 None None] MFA Rules not processed for user `befc6c8959484a18b0a0a4f01a6bb44d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:12.686312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0194e161-c5ac-45b3-8f9f-af1eb01fafb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:12.847989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 909/1820] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:12.885790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 912/1821] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:12.891447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b63fd-ad92-4e6e-9f23-25a4df205e71 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:12.905894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e161-c5ac-45b3-8f9f-af1eb01fafb0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:12.909975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b63fd-ad92-4e6e-9f23-25a4df205e71 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:12.911925 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0194e161-c5ac-45b3-8f9f-af1eb01fafb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:12.930852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f3d1c-6946-483d-a8dd-6a60398cd99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:12.932385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f3d1c-6946-483d-a8dd-6a60398cd99e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:12.932962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f3d1c-6946-483d-a8dd-6a60398cd99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:12.933332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f3d1c-6946-483d-a8dd-6a60398cd99e admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:12.934331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f3d1c-6946-483d-a8dd-6a60398cd99e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:12.939223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f3d1c-6946-483d-a8dd-6a60398cd99e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:12.988114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e161-c5ac-45b3-8f9f-af1eb01fafb0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:13.011960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c98a04-8c20-42ab-b586-b9846093c9dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:13.012590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98a04-8c20-42ab-b586-b9846093c9dc service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:13.014276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98a04-8c20-42ab-b586-b9846093c9dc service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:13.015451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98a04-8c20-42ab-b586-b9846093c9dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:13.039011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 910/1822] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:12 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:13.043053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f3d1c-6946-483d-a8dd-6a60398cd99e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:13.054824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c98a04-8c20-42ab-b586-b9846093c9dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:13.056536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c98a04-8c20-42ab-b586-b9846093c9dc service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:13.076299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 913/1823] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:13.145639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f3d1c-6946-483d-a8dd-6a60398cd99e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:13.151264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa32eca-9625-4af7-9478-78910b8cfecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:13.152072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32eca-9625-4af7-9478-78910b8cfecf admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:13.152675 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32eca-9625-4af7-9478-78910b8cfecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:13.153070 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32eca-9625-4af7-9478-78910b8cfecf admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:13.153811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa32eca-9625-4af7-9478-78910b8cfecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:13.155931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa32eca-9625-4af7-9478-78910b8cfecf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:13.186117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 911/1824] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:13 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:13.190550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c98a04-8c20-42ab-b586-b9846093c9dc service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:13.202732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c98a04-8c20-42ab-b586-b9846093c9dc service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:13.207747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e889f3-757e-4f9d-bfc2-a165b1ae3573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:13.208374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e889f3-757e-4f9d-bfc2-a165b1ae3573 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:13.208650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e889f3-757e-4f9d-bfc2-a165b1ae3573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:13.208923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e889f3-757e-4f9d-bfc2-a165b1ae3573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:13.209571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e889f3-757e-4f9d-bfc2-a165b1ae3573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:13.211194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e889f3-757e-4f9d-bfc2-a165b1ae3573 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:13.225773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 914/1825] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:13.227435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa32eca-9625-4af7-9478-78910b8cfecf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:13.238454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa32eca-9625-4af7-9478-78910b8cfecf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:13.243883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc0544-dc5f-4f9b-8cd7-284c5c4d1fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:13.244581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0544-dc5f-4f9b-8cd7-284c5c4d1fea admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:13.244905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0544-dc5f-4f9b-8cd7-284c5c4d1fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:13.245169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0544-dc5f-4f9b-8cd7-284c5c4d1fea admin admin] PATH_INFO: `/v3/projects/eacd0759bc654e1b9fffbc0f67f34a5d/users/6ca1df33e0204a6cba9501a181561402/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:13.262238 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc0544-dc5f-4f9b-8cd7-284c5c4d1fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacd0759bc654e1b9fffbc0f67f34a5d, user_id=6ca1df33e0204a6cba9501a181561402, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:13.264945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc0544-dc5f-4f9b-8cd7-284c5c4d1fea admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:13.280280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 912/1826] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:13 2022] PUT /identity/v3/projects/eacd0759bc654e1b9fffbc0f67f34a5d/users/6ca1df33e0204a6cba9501a181561402/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:13.284545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e889f3-757e-4f9d-bfc2-a165b1ae3573 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:13.295171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e889f3-757e-4f9d-bfc2-a165b1ae3573 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:13.358854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365ac26-9fcf-4887-a81f-79fe0b0fca0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:13.359605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365ac26-9fcf-4887-a81f-79fe0b0fca0e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:13.359882 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365ac26-9fcf-4887-a81f-79fe0b0fca0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:13.360150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365ac26-9fcf-4887-a81f-79fe0b0fca0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:13.360794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365ac26-9fcf-4887-a81f-79fe0b0fca0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:13.362451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365ac26-9fcf-4887-a81f-79fe0b0fca0e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:13.376941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 915/1827] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:13.381539 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc0544-dc5f-4f9b-8cd7-284c5c4d1fea admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:13.394564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc0544-dc5f-4f9b-8cd7-284c5c4d1fea admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:13.399715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:13.400473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:13.401891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:13.401891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] PATH_INFO: `/v3/projects/eacd0759bc654e1b9fffbc0f67f34a5d/users/6ca1df33e0204a6cba9501a181561402/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:13.403582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacd0759bc654e1b9fffbc0f67f34a5d, user_id=6ca1df33e0204a6cba9501a181561402, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:13.406546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:13.428763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 913/1828] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:13 2022] PUT /identity/v3/projects/eacd0759bc654e1b9fffbc0f67f34a5d/users/6ca1df33e0204a6cba9501a181561402/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:14.224746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d0fd8-c192-4f73-a653-ed0cd346b405 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:14.225304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d0fd8-c192-4f73-a653-ed0cd346b405 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:14.225304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d0fd8-c192-4f73-a653-ed0cd346b405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:14.254057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d30d0fd8-c192-4f73-a653-ed0cd346b405 None None] MFA Rules not processed for user `6ca1df33e0204a6cba9501a181561402`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:14.255408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d30d0fd8-c192-4f73-a653-ed0cd346b405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:14.261125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:14.333994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 916/1829] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:14.346519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:14.348885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:14.395284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:14.413890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e84dc-3088-4b92-8713-07c2e00f67ef admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:14.417681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff69c-eaca-46b6-b6cb-e1e1849177bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:14.418322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff69c-eaca-46b6-b6cb-e1e1849177bc service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:14.418603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff69c-eaca-46b6-b6cb-e1e1849177bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:14.418875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff69c-eaca-46b6-b6cb-e1e1849177bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:14.486945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ff69c-eaca-46b6-b6cb-e1e1849177bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:14.488454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ff69c-eaca-46b6-b6cb-e1e1849177bc service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:14.500052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 914/1830] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:15.992007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d0fd8-c192-4f73-a653-ed0cd346b405 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:15.994006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d30d0fd8-c192-4f73-a653-ed0cd346b405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:16.013849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d0fd8-c192-4f73-a653-ed0cd346b405 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:16.027003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d07c0-7202-4e87-abe8-e79c06cb1e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:16.027749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d07c0-7202-4e87-abe8-e79c06cb1e43 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:16.028067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d07c0-7202-4e87-abe8-e79c06cb1e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:16.028522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d07c0-7202-4e87-abe8-e79c06cb1e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:16.046193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d07c0-7202-4e87-abe8-e79c06cb1e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:16.048689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d07c0-7202-4e87-abe8-e79c06cb1e43 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:16.061559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 917/1831] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:16.578684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff69c-eaca-46b6-b6cb-e1e1849177bc service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:16.595000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff69c-eaca-46b6-b6cb-e1e1849177bc service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:16.601228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb77ea-6736-4d17-89e2-dbedff69a945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:16.602396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb77ea-6736-4d17-89e2-dbedff69a945 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:16.602975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb77ea-6736-4d17-89e2-dbedff69a945 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:16.603495 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb77ea-6736-4d17-89e2-dbedff69a945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:16.616542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-89bb77ea-6736-4d17-89e2-dbedff69a945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:16.728537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb77ea-6736-4d17-89e2-dbedff69a945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:16.733712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb77ea-6736-4d17-89e2-dbedff69a945 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:16.753417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 915/1832] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:16.820779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27177b-0fe1-4b34-a3dc-f836db2cfc8e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:16.821295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27177b-0fe1-4b34-a3dc-f836db2cfc8e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:16.821569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27177b-0fe1-4b34-a3dc-f836db2cfc8e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:16.823211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 918/1833] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:19:16.849853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d912a0-8e16-4a08-9d1f-3b0a0d115a5b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:16.850207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d912a0-8e16-4a08-9d1f-3b0a0d115a5b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:16.850797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d912a0-8e16-4a08-9d1f-3b0a0d115a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:16.893561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-90d912a0-8e16-4a08-9d1f-3b0a0d115a5b None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:16.896978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-90d912a0-8e16-4a08-9d1f-3b0a0d115a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:17.010984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 916/1834] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:19:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:17.080405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27177b-0fe1-4b34-a3dc-f836db2cfc8e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:17.112116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27177b-0fe1-4b34-a3dc-f836db2cfc8e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:17.119369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c6d90-9d4e-4f02-9044-080fdbd3ed8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:17.120017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c6d90-9d4e-4f02-9044-080fdbd3ed8b service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:17.120292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c6d90-9d4e-4f02-9044-080fdbd3ed8b service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:17.120555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c6d90-9d4e-4f02-9044-080fdbd3ed8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:17.130037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c6d90-9d4e-4f02-9044-080fdbd3ed8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:17.131659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c6d90-9d4e-4f02-9044-080fdbd3ed8b service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:17.142646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 919/1835] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:19.221068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d912a0-8e16-4a08-9d1f-3b0a0d115a5b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:19.222651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-90d912a0-8e16-4a08-9d1f-3b0a0d115a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:19.294452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:19.448399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d912a0-8e16-4a08-9d1f-3b0a0d115a5b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:19.470829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:19.471587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:19.471905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:19.472144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] PATH_INFO: `/v3/users/b5277af7465e4b87ae4ce17e1010a98b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:19.476541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5277af7465e4b87ae4ce17e1010a98b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:19.481982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:19.600788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5277af7465e4b87ae4ce17e1010a98b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:19.602459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5277af7465e4b87ae4ce17e1010a98b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:19.621375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5277af7465e4b87ae4ce17e1010a98b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:19.644896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5277af7465e4b87ae4ce17e1010a98b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:19.656388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5277af7465e4b87ae4ce17e1010a98b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:19.674978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 917/1836] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:19 2022] DELETE /identity/v3/users/b5277af7465e4b87ae4ce17e1010a98b => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:19.679047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645262b-a1e2-4722-b02c-0c519d1c614f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:19.679601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645262b-a1e2-4722-b02c-0c519d1c614f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:19.679859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645262b-a1e2-4722-b02c-0c519d1c614f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:19.703038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a645262b-a1e2-4722-b02c-0c519d1c614f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:19.704998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a645262b-a1e2-4722-b02c-0c519d1c614f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:19.774921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 920/1837] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:19.792126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:19.793730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:19.862042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c43d1b-344b-4828-92dd-05fc0e4a5dc3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:19.865839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:19.866651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:19.866935 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:19.867208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:19.886796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:19.892800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:19.909806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 918/1838] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:20.356542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645262b-a1e2-4722-b02c-0c519d1c614f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:20.358197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a645262b-a1e2-4722-b02c-0c519d1c614f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:20.373973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645262b-a1e2-4722-b02c-0c519d1c614f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:20.380331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:20.381632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:20.382056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:20.382482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] PATH_INFO: `/v3/users/038df7efeb0f4291912c0040ec90deb0` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:20.384070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] RBAC: Authorizing `identity:delete_user(user_id=038df7efeb0f4291912c0040ec90deb0)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:20.386003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:20.489094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038df7efeb0f4291912c0040ec90deb0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.489915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038df7efeb0f4291912c0040ec90deb0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.502550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038df7efeb0f4291912c0040ec90deb0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.505614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038df7efeb0f4291912c0040ec90deb0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.506585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:20.509745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:20.527184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0046a78-f46e-4296-936d-5c2d2531558f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038df7efeb0f4291912c0040ec90deb0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.538127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:20.547282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632d074-5239-46ca-9876-50e0810208ae service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:20.565131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:20.565790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:20.566353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:20.566636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] PATH_INFO: `/v3/projects/c020af9ba5cc41aa878027184cd419ae` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:20.568798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c020af9ba5cc41aa878027184cd419ae)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:20.571832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:20.573261 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 921/1839] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:20 2022] DELETE /identity/v3/users/038df7efeb0f4291912c0040ec90deb0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:20.577450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37092f73-b6f5-48b4-8a8a-cb42cbb78b26 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:20.578250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37092f73-b6f5-48b4-8a8a-cb42cbb78b26 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:20.578250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37092f73-b6f5-48b4-8a8a-cb42cbb78b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:20.617174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-37092f73-b6f5-48b4-8a8a-cb42cbb78b26 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:20.619803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-37092f73-b6f5-48b4-8a8a-cb42cbb78b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:20.632369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:20.647087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c020af9ba5cc41aa878027184cd419ae', 'request_id': 'req-cb22116e-f1e1-4437-a95f-c4697f9bea3e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.658287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c020af9ba5cc41aa878027184cd419ae', 'request_id': 'req-cb22116e-f1e1-4437-a95f-c4697f9bea3e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.675944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c020af9ba5cc41aa878027184cd419ae', 'request_id': 'req-cb22116e-f1e1-4437-a95f-c4697f9bea3e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.722891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] The token cache is being invalidate because project c020af9ba5cc41aa878027184cd419ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:20.723484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.725127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.726804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 922/1840] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:20.728826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 919/1841] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:20 2022] DELETE /identity/v3/projects/c020af9ba5cc41aa878027184cd419ae => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:20.738840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:20.739284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37092f73-b6f5-48b4-8a8a-cb42cbb78b26 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:20.740809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-37092f73-b6f5-48b4-8a8a-cb42cbb78b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:20.740929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:20.757662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37092f73-b6f5-48b4-8a8a-cb42cbb78b26 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:20.896433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:20.897102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:20.897381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:20.897654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] PATH_INFO: `/v3/users/41e3cec3a49d484d81a0aca4758635c4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:20.899821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e3cec3a49d484d81a0aca4758635c4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:20.901780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:20.904738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22116e-f1e1-4437-a95f-c4697f9bea3e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:20.911293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bb6a1-f5e2-49a7-b181-01118448571e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:20.912079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bb6a1-f5e2-49a7-b181-01118448571e service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:20.912458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bb6a1-f5e2-49a7-b181-01118448571e service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:20.912828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bb6a1-f5e2-49a7-b181-01118448571e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:20.915473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0a2bb6a1-f5e2-49a7-b181-01118448571e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:20.936726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bb6a1-f5e2-49a7-b181-01118448571e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:20.938641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bb6a1-f5e2-49a7-b181-01118448571e service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:20.967300 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 920/1842] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:20.991455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e3cec3a49d484d81a0aca4758635c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:20.994370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e3cec3a49d484d81a0aca4758635c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.015265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e3cec3a49d484d81a0aca4758635c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.016590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e3cec3a49d484d81a0aca4758635c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.045687 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe6e2537-c503-49e9-a879-e0d45759ee5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e3cec3a49d484d81a0aca4758635c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.069913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 923/1843] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:20 2022] DELETE /identity/v3/users/41e3cec3a49d484d81a0aca4758635c4 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:21.272339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e663-9d59-485b-b876-f14914ec2f3c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:21.272630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e663-9d59-485b-b876-f14914ec2f3c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:21.272917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e663-9d59-485b-b876-f14914ec2f3c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:21.274284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 921/1844] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:19:21.291049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895943d5-a89c-4820-9efa-6015429a904f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:21.291783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895943d5-a89c-4820-9efa-6015429a904f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:21.292331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895943d5-a89c-4820-9efa-6015429a904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:21.329774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-895943d5-a89c-4820-9efa-6015429a904f None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:21.333204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-895943d5-a89c-4820-9efa-6015429a904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:21.336719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:21.488914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e663-9d59-485b-b876-f14914ec2f3c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:21.491554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6f46e663-9d59-485b-b876-f14914ec2f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:21.565453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 924/1845] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:19:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:21.671113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e663-9d59-485b-b876-f14914ec2f3c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:21.677503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:21.678277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:21.678651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:21.679029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] PATH_INFO: `/v3/projects/fa2b2d0349af490e88cd539f7a0a2f92` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:21.680476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2b2d0349af490e88cd539f7a0a2f92)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:21.683027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:21.750221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2b2d0349af490e88cd539f7a0a2f92', 'request_id': 'req-712e9720-5427-4075-905c-1d77d9606a96'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.757714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2b2d0349af490e88cd539f7a0a2f92', 'request_id': 'req-712e9720-5427-4075-905c-1d77d9606a96'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.765931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2b2d0349af490e88cd539f7a0a2f92', 'request_id': 'req-712e9720-5427-4075-905c-1d77d9606a96'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.783446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:21.797111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] The token cache is being invalidate because project fa2b2d0349af490e88cd539f7a0a2f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:21.798260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.800601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:21.804336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 922/1846] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:21 2022] DELETE /identity/v3/projects/fa2b2d0349af490e88cd539f7a0a2f92 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:21.808812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895943d5-a89c-4820-9efa-6015429a904f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:21.823464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895943d5-a89c-4820-9efa-6015429a904f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:21.910254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:21.911103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:21.911424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:21.911696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] PATH_INFO: `/v3/users/b7c5b2d188914ffcaa9d7fc2ff491181` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:21.913611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c5b2d188914ffcaa9d7fc2ff491181)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:21.915911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:21.976884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:21.978807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:22.057262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c5b2d188914ffcaa9d7fc2ff491181'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.057540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c5b2d188914ffcaa9d7fc2ff491181'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.077237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c5b2d188914ffcaa9d7fc2ff491181'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.078043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c5b2d188914ffcaa9d7fc2ff491181'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.113984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c5b2d188914ffcaa9d7fc2ff491181'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.154631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 925/1847] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:21 2022] DELETE /identity/v3/users/b7c5b2d188914ffcaa9d7fc2ff491181 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:22.154631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:22.154631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:22.230226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9720-5427-4075-905c-1d77d9606a96 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:22.238098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:22.355151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55869dc5-ae3f-4be8-9f78-64f9d5b5df4b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:22.367904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:22.368603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:22.368880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:22.369150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:22.369908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:22.371255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:22.372697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:22.373139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:22.373918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] PATH_INFO: `/v3/projects/d3a18c25c7634825ac2365b48dd5d208` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:22.379802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a18c25c7634825ac2365b48dd5d208)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:22.383111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:22.388855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:22.393467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:22.407603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 923/1848] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:22.517817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a18c25c7634825ac2365b48dd5d208', 'request_id': 'req-b760653f-1813-4222-b72b-c8b3baf4aa14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.533819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a18c25c7634825ac2365b48dd5d208', 'request_id': 'req-b760653f-1813-4222-b72b-c8b3baf4aa14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.551656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a18c25c7634825ac2365b48dd5d208', 'request_id': 'req-b760653f-1813-4222-b72b-c8b3baf4aa14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.583370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] The token cache is being invalidate because project d3a18c25c7634825ac2365b48dd5d208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:22.584310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.589129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.596299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 926/1849] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:22 2022] DELETE /identity/v3/projects/d3a18c25c7634825ac2365b48dd5d208 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:22.596888 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:22.620709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba668d-a42b-40cf-8aba-808cfb879533 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:22.728404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:22.729847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:22.730285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:22.730711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] PATH_INFO: `/v3/users/89f6116f611d4b49bdf66dd74d7d7d8e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:22.734001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f6116f611d4b49bdf66dd74d7d7d8e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:22.738302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:22.776865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:22.839485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f6116f611d4b49bdf66dd74d7d7d8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.840210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f6116f611d4b49bdf66dd74d7d7d8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.876974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f6116f611d4b49bdf66dd74d7d7d8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.889787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f6116f611d4b49bdf66dd74d7d7d8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.913813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-02a5fa9f-3681-4a45-bc41-eeace7e4c879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f6116f611d4b49bdf66dd74d7d7d8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:22.931039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 924/1850] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:22 2022] DELETE /identity/v3/users/89f6116f611d4b49bdf66dd74d7d7d8e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:22.979583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:22.981158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:23.006921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760653f-1813-4222-b72b-c8b3baf4aa14 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:23.127991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:23.128878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.129333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.129780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] PATH_INFO: `/v3/projects/792bd1fe023b42e5a94f36c7d47b0889` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.131724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=792bd1fe023b42e5a94f36c7d47b0889)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:23.134447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:23.196057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792bd1fe023b42e5a94f36c7d47b0889', 'request_id': 'req-89869621-b26f-4c16-8791-176fbb6388b9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.198791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:23.205032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792bd1fe023b42e5a94f36c7d47b0889', 'request_id': 'req-89869621-b26f-4c16-8791-176fbb6388b9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.216382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792bd1fe023b42e5a94f36c7d47b0889', 'request_id': 'req-89869621-b26f-4c16-8791-176fbb6388b9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.232562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] The token cache is being invalidate because project 792bd1fe023b42e5a94f36c7d47b0889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:23.233034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.233902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-89869621-b26f-4c16-8791-176fbb6388b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.237986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 927/1851] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:22 2022] DELETE /identity/v3/projects/792bd1fe023b42e5a94f36c7d47b0889 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:23.241644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11984f79-07cd-4da9-82c2-194d6d22e9ac None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.242029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11984f79-07cd-4da9-82c2-194d6d22e9ac None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.244918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11984f79-07cd-4da9-82c2-194d6d22e9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.281757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-11984f79-07cd-4da9-82c2-194d6d22e9ac None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:23.283649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-11984f79-07cd-4da9-82c2-194d6d22e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:23.421745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 925/1852] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:23.439620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e0ce-707f-41ea-afab-5b545f4ec432 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.439908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e0ce-707f-41ea-afab-5b545f4ec432 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.440185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e0ce-707f-41ea-afab-5b545f4ec432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.469079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-51c8e0ce-707f-41ea-afab-5b545f4ec432 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:23.470567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-51c8e0ce-707f-41ea-afab-5b545f4ec432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:23.487415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 928/1853] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:23.488183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11984f79-07cd-4da9-82c2-194d6d22e9ac None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:23.506155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11984f79-07cd-4da9-82c2-194d6d22e9ac None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:23.517428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:23.517981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.518259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.518527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.519918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:23.520376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:23.522023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:23.556972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 926/1854] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:23.558426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8e0ce-707f-41ea-afab-5b545f4ec432 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:23.565672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:23.568009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:23.575453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8e0ce-707f-41ea-afab-5b545f4ec432 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:23.581607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a43958-9d5e-48ad-ba30-f5f982a5724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:23.582849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a43958-9d5e-48ad-ba30-f5f982a5724f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.583460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a43958-9d5e-48ad-ba30-f5f982a5724f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.583935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a43958-9d5e-48ad-ba30-f5f982a5724f admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.584846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a43958-9d5e-48ad-ba30-f5f982a5724f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:23.587274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a43958-9d5e-48ad-ba30-f5f982a5724f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:23.597536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f7ff-26de-4df1-871a-1079893a3fcb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:23.611411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:23.613998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.614598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.615802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] PATH_INFO: `/v3/projects/ede043b2d6994d10b3d6111fc8bf4686` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.620079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede043b2d6994d10b3d6111fc8bf4686)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:23.627019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:23.631954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 929/1855] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:23 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:23.633104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a43958-9d5e-48ad-ba30-f5f982a5724f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:23.722833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a43958-9d5e-48ad-ba30-f5f982a5724f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:23.727568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede043b2d6994d10b3d6111fc8bf4686', 'request_id': 'req-786e64e1-4516-4969-be31-3c5c8d837f84'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.736543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede043b2d6994d10b3d6111fc8bf4686', 'request_id': 'req-786e64e1-4516-4969-be31-3c5c8d837f84'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.746400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede043b2d6994d10b3d6111fc8bf4686', 'request_id': 'req-786e64e1-4516-4969-be31-3c5c8d837f84'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.770704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] The token cache is being invalidate because project ede043b2d6994d10b3d6111fc8bf4686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:23.771155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.772817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-786e64e1-4516-4969-be31-3c5c8d837f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:23.775692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 927/1856] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:23 2022] DELETE /identity/v3/projects/ede043b2d6994d10b3d6111fc8bf4686 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:23.783311 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60d727-6aa3-4722-9eef-478de26ce3cc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.784024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60d727-6aa3-4722-9eef-478de26ce3cc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.784267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60d727-6aa3-4722-9eef-478de26ce3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.846366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7a60d727-6aa3-4722-9eef-478de26ce3cc None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:23.848009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7a60d727-6aa3-4722-9eef-478de26ce3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:23.865236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:23.882595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:23.883146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.883485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.883757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.885070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:23.888431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:23.919560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 930/1857] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:23 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:23.936520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:23.938665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:23.950834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66425116-8f20-4d87-8217-5352821f7ef7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:23.956857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad47c8e-564d-41f0-b849-b8aa14b93894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:23.957483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad47c8e-564d-41f0-b849-b8aa14b93894 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.957785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad47c8e-564d-41f0-b849-b8aa14b93894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.958056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad47c8e-564d-41f0-b849-b8aa14b93894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.958694 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad47c8e-564d-41f0-b849-b8aa14b93894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:23.960330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad47c8e-564d-41f0-b849-b8aa14b93894 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:23.972364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 931/1858] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:23.976350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad47c8e-564d-41f0-b849-b8aa14b93894 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:23.987833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 928/1859] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:23.988298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad47c8e-564d-41f0-b849-b8aa14b93894 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:23.991818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e200fb2-e253-4c3c-83c5-3ec1cc86c2d7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.992111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e200fb2-e253-4c3c-83c5-3ec1cc86c2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.992540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e200fb2-e253-4c3c-83c5-3ec1cc86c2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.993749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedafaf-f123-42b7-8f17-e8d042446f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:23.994658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedafaf-f123-42b7-8f17-e8d042446f2c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:23.995030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedafaf-f123-42b7-8f17-e8d042446f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:23.995441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedafaf-f123-42b7-8f17-e8d042446f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:23.996198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedafaf-f123-42b7-8f17-e8d042446f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:23.998239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedafaf-f123-42b7-8f17-e8d042446f2c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.014631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 932/1860] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:24.016021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedafaf-f123-42b7-8f17-e8d042446f2c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.019507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6e200fb2-e253-4c3c-83c5-3ec1cc86c2d7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:24.020774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6e200fb2-e253-4c3c-83c5-3ec1cc86c2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:24.028237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedafaf-f123-42b7-8f17-e8d042446f2c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.029293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 929/1861] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:24.033696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e200fb2-e253-4c3c-83c5-3ec1cc86c2d7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.034299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca79e9-271b-4a37-8fe4-714645157f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.035074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca79e9-271b-4a37-8fe4-714645157f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.035475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca79e9-271b-4a37-8fe4-714645157f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.035840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca79e9-271b-4a37-8fe4-714645157f19 admin admin] PATH_INFO: `/v3/projects/74abe8f121154e0796727b9d3a3444d6/users/96f04be8ff2842bfad34cea858171b77/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.047766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e200fb2-e253-4c3c-83c5-3ec1cc86c2d7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.054845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7f18a-c420-4abe-9305-1a41c784cd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.055602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7f18a-c420-4abe-9305-1a41c784cd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.055961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7f18a-c420-4abe-9305-1a41c784cd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.056286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7f18a-c420-4abe-9305-1a41c784cd54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.057090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7f18a-c420-4abe-9305-1a41c784cd54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:24.057435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7f18a-c420-4abe-9305-1a41c784cd54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.058719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca79e9-271b-4a37-8fe4-714645157f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74abe8f121154e0796727b9d3a3444d6, user_id=96f04be8ff2842bfad34cea858171b77, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.059383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7f18a-c420-4abe-9305-1a41c784cd54 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.062139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca79e9-271b-4a37-8fe4-714645157f19 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.080781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 933/1862] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:24 2022] PUT /identity/v3/projects/74abe8f121154e0796727b9d3a3444d6/users/96f04be8ff2842bfad34cea858171b77/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:24.085691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 930/1863] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:19:24.088234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca79e9-271b-4a37-8fe4-714645157f19 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.091704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2790ea-6b56-4ddc-a91c-87a33b02844f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.092219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2790ea-6b56-4ddc-a91c-87a33b02844f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.092688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2790ea-6b56-4ddc-a91c-87a33b02844f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.093453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:24.144641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fa2790ea-6b56-4ddc-a91c-87a33b02844f None None] MFA Rules not processed for user `96f04be8ff2842bfad34cea858171b77`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:24.145862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fa2790ea-6b56-4ddc-a91c-87a33b02844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:24.174588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca79e9-271b-4a37-8fe4-714645157f19 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.181036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6571a2-8b3f-47a8-952f-788294934b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.181745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6571a2-8b3f-47a8-952f-788294934b00 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.182083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6571a2-8b3f-47a8-952f-788294934b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.182424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6571a2-8b3f-47a8-952f-788294934b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.183124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6571a2-8b3f-47a8-952f-788294934b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.185217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6571a2-8b3f-47a8-952f-788294934b00 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.223031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 934/1864] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:24.227470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6571a2-8b3f-47a8-952f-788294934b00 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.234162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 931/1865] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:24.238800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2790ea-6b56-4ddc-a91c-87a33b02844f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.249705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2790ea-6b56-4ddc-a91c-87a33b02844f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.294848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6571a2-8b3f-47a8-952f-788294934b00 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.300298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76160c8c-808b-4765-a472-f4e928fed634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.300912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160c8c-808b-4765-a472-f4e928fed634 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.301186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160c8c-808b-4765-a472-f4e928fed634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.301456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160c8c-808b-4765-a472-f4e928fed634 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.302232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76160c8c-808b-4765-a472-f4e928fed634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.304197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76160c8c-808b-4765-a472-f4e928fed634 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.313451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde8236-a6f6-4827-9d40-3755c6e37bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.314155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde8236-a6f6-4827-9d40-3755c6e37bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.314502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde8236-a6f6-4827-9d40-3755c6e37bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.314839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde8236-a6f6-4827-9d40-3755c6e37bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.315639 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde8236-a6f6-4827-9d40-3755c6e37bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.318034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde8236-a6f6-4827-9d40-3755c6e37bb7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.333797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 935/1866] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:24.335338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76160c8c-808b-4765-a472-f4e928fed634 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.351361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76160c8c-808b-4765-a472-f4e928fed634 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.353877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 932/1867] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:24.357613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde8236-a6f6-4827-9d40-3755c6e37bb7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.370738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:24.427697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783f27-0fde-45b1-9db6-2017942bcd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.428330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783f27-0fde-45b1-9db6-2017942bcd44 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.428610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783f27-0fde-45b1-9db6-2017942bcd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.429173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783f27-0fde-45b1-9db6-2017942bcd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.429829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5783f27-0fde-45b1-9db6-2017942bcd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.431035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde8236-a6f6-4827-9d40-3755c6e37bb7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.431497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5783f27-0fde-45b1-9db6-2017942bcd44 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.436090 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388bdaf-bbbf-4b3e-af4e-afeab78608b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.436737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388bdaf-bbbf-4b3e-af4e-afeab78608b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.437018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388bdaf-bbbf-4b3e-af4e-afeab78608b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.437293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388bdaf-bbbf-4b3e-af4e-afeab78608b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.438116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388bdaf-bbbf-4b3e-af4e-afeab78608b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.440099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388bdaf-bbbf-4b3e-af4e-afeab78608b5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.443284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 936/1868] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:24.447295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783f27-0fde-45b1-9db6-2017942bcd44 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.459655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783f27-0fde-45b1-9db6-2017942bcd44 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.463350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 933/1869] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:24.467189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388bdaf-bbbf-4b3e-af4e-afeab78608b5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.471986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce82bf-2c98-40d2-86b1-8640411b650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.472738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce82bf-2c98-40d2-86b1-8640411b650c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.473118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce82bf-2c98-40d2-86b1-8640411b650c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.473540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce82bf-2c98-40d2-86b1-8640411b650c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.474504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce82bf-2c98-40d2-86b1-8640411b650c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.476317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce82bf-2c98-40d2-86b1-8640411b650c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.478427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388bdaf-bbbf-4b3e-af4e-afeab78608b5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.485619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d83bd-50f2-408b-a1b7-438ca9fdb3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.486367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d83bd-50f2-408b-a1b7-438ca9fdb3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.486640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d83bd-50f2-408b-a1b7-438ca9fdb3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.486911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d83bd-50f2-408b-a1b7-438ca9fdb3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.487589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d83bd-50f2-408b-a1b7-438ca9fdb3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.489003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 937/1870] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:24.489255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d83bd-50f2-408b-a1b7-438ca9fdb3c2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.493236 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce82bf-2c98-40d2-86b1-8640411b650c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.498281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:24.503937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 934/1871] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:24.505067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d83bd-50f2-408b-a1b7-438ca9fdb3c2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.508433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce82bf-2c98-40d2-86b1-8640411b650c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.513631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ab08-4459-455a-a2d9-8cfd3d78a800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.514283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ab08-4459-455a-a2d9-8cfd3d78a800 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.514573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ab08-4459-455a-a2d9-8cfd3d78a800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.514937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ab08-4459-455a-a2d9-8cfd3d78a800 admin admin] PATH_INFO: `/v3/projects/83dae260602c4a0288449110410a4337/users/c5ac3ea0e7214ba0b47fb170e56e6db9/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.515335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d83bd-50f2-408b-a1b7-438ca9fdb3c2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.520229 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4b298-4742-4602-9003-857f60cbf151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.520884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4b298-4742-4602-9003-857f60cbf151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.521174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4b298-4742-4602-9003-857f60cbf151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.521412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4b298-4742-4602-9003-857f60cbf151 admin admin] PATH_INFO: `/v3/projects/a2ad200ec2ec4658aa077ef60b57ac2e/users/4ce06e613d96442ea36d1c522c9da578/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.530537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735ab08-4459-455a-a2d9-8cfd3d78a800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83dae260602c4a0288449110410a4337, user_id=c5ac3ea0e7214ba0b47fb170e56e6db9, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.533222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735ab08-4459-455a-a2d9-8cfd3d78a800 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.536405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4b298-4742-4602-9003-857f60cbf151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ad200ec2ec4658aa077ef60b57ac2e, user_id=4ce06e613d96442ea36d1c522c9da578, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.539223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4b298-4742-4602-9003-857f60cbf151 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.546280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 938/1872] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:24 2022] PUT /identity/v3/projects/83dae260602c4a0288449110410a4337/users/c5ac3ea0e7214ba0b47fb170e56e6db9/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:24.552079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 935/1873] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:24 2022] PUT /identity/v3/projects/a2ad200ec2ec4658aa077ef60b57ac2e/users/4ce06e613d96442ea36d1c522c9da578/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:24.555932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ab08-4459-455a-a2d9-8cfd3d78a800 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.559219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e97e-60a1-4e28-8950-97d1813cf8e4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.559522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e97e-60a1-4e28-8950-97d1813cf8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.559788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e97e-60a1-4e28-8950-97d1813cf8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.587464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1a79e97e-60a1-4e28-8950-97d1813cf8e4 None None] MFA Rules not processed for user `c5ac3ea0e7214ba0b47fb170e56e6db9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:24.589178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1a79e97e-60a1-4e28-8950-97d1813cf8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:24.620306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ab08-4459-455a-a2d9-8cfd3d78a800 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.658689 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 936/1874] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:24.663347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e97e-60a1-4e28-8950-97d1813cf8e4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.684042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74ad8f-8a39-44fd-b472-dadb335b33b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.684671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74ad8f-8a39-44fd-b472-dadb335b33b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.684968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74ad8f-8a39-44fd-b472-dadb335b33b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.685224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74ad8f-8a39-44fd-b472-dadb335b33b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.685874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74ad8f-8a39-44fd-b472-dadb335b33b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.687952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74ad8f-8a39-44fd-b472-dadb335b33b5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.698569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 939/1875] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:24.702699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74ad8f-8a39-44fd-b472-dadb335b33b5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.712744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74ad8f-8a39-44fd-b472-dadb335b33b5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.718033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:24.718555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4c417-6f9d-421a-b2ce-a6f29d22cbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.719216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4c417-6f9d-421a-b2ce-a6f29d22cbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.719559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4c417-6f9d-421a-b2ce-a6f29d22cbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.719860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4c417-6f9d-421a-b2ce-a6f29d22cbd5 admin admin] PATH_INFO: `/v3/projects/a2ad200ec2ec4658aa077ef60b57ac2e/users/4ce06e613d96442ea36d1c522c9da578/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.722040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4c417-6f9d-421a-b2ce-a6f29d22cbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ad200ec2ec4658aa077ef60b57ac2e, user_id=4ce06e613d96442ea36d1c522c9da578, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.724731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4c417-6f9d-421a-b2ce-a6f29d22cbd5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.731146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e97e-60a1-4e28-8950-97d1813cf8e4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.737529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 940/1876] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:24 2022] PUT /identity/v3/projects/a2ad200ec2ec4658aa077ef60b57ac2e/users/4ce06e613d96442ea36d1c522c9da578/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:24.748952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d6db4-34d7-4baa-8d23-e3c68b6af936 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.749713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d6db4-34d7-4baa-8d23-e3c68b6af936 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.749713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d6db4-34d7-4baa-8d23-e3c68b6af936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.777929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f22d6db4-34d7-4baa-8d23-e3c68b6af936 None None] MFA Rules not processed for user `4ce06e613d96442ea36d1c522c9da578`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:24.779503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f22d6db4-34d7-4baa-8d23-e3c68b6af936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:24.792709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7c48-f090-460b-b994-ad34a7d0f544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.793725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7c48-f090-460b-b994-ad34a7d0f544 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.794180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7c48-f090-460b-b994-ad34a7d0f544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.794615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7c48-f090-460b-b994-ad34a7d0f544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.795683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b7c48-f090-460b-b994-ad34a7d0f544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.799025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b7c48-f090-460b-b994-ad34a7d0f544 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.833286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 937/1877] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:24.837705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7c48-f090-460b-b994-ad34a7d0f544 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.852343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:24.867130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 941/1878] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:24.881730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d6db4-34d7-4baa-8d23-e3c68b6af936 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.883782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f22d6db4-34d7-4baa-8d23-e3c68b6af936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:24.929854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7c48-f090-460b-b994-ad34a7d0f544 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.935953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d5cba-caeb-49d2-8785-6ff7ccb184ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:24.937276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d5cba-caeb-49d2-8785-6ff7ccb184ec admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:24.937737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d5cba-caeb-49d2-8785-6ff7ccb184ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:24.938148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d5cba-caeb-49d2-8785-6ff7ccb184ec admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:24.940958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d5cba-caeb-49d2-8785-6ff7ccb184ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:24.942206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d5cba-caeb-49d2-8785-6ff7ccb184ec admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:24.958247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d6db4-34d7-4baa-8d23-e3c68b6af936 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:24.976087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 938/1879] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:24 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:24.983920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d5cba-caeb-49d2-8785-6ff7ccb184ec admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:24.996798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d5cba-caeb-49d2-8785-6ff7ccb184ec admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.003850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa29b-7eff-4987-b189-15c355db71b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.004485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa29b-7eff-4987-b189-15c355db71b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.004772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa29b-7eff-4987-b189-15c355db71b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.005044 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa29b-7eff-4987-b189-15c355db71b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.005701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7aa29b-7eff-4987-b189-15c355db71b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.007559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7aa29b-7eff-4987-b189-15c355db71b6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.023506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 939/1880] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:25.024910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa29b-7eff-4987-b189-15c355db71b6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:25.032416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81439050-eb73-4fc9-a04a-d789844eddd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.033072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81439050-eb73-4fc9-a04a-d789844eddd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.033401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81439050-eb73-4fc9-a04a-d789844eddd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.033729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81439050-eb73-4fc9-a04a-d789844eddd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.037067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa29b-7eff-4987-b189-15c355db71b6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.047578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0ed24-7a38-4e85-8968-ee437b1792bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.048416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0ed24-7a38-4e85-8968-ee437b1792bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.048708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0ed24-7a38-4e85-8968-ee437b1792bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.048969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0ed24-7a38-4e85-8968-ee437b1792bb admin admin] PATH_INFO: `/v3/projects/077820bebf51491a87a466ab6403b86a/users/c53f2b3a14b9497e9fef8ef02535a96f/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.068446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0ed24-7a38-4e85-8968-ee437b1792bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=077820bebf51491a87a466ab6403b86a, user_id=c53f2b3a14b9497e9fef8ef02535a96f, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.071699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0ed24-7a38-4e85-8968-ee437b1792bb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.093306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 940/1881] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:25 2022] PUT /identity/v3/projects/077820bebf51491a87a466ab6403b86a/users/c53f2b3a14b9497e9fef8ef02535a96f/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:25.097723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0ed24-7a38-4e85-8968-ee437b1792bb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:25.140955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81439050-eb73-4fc9-a04a-d789844eddd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.142800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81439050-eb73-4fc9-a04a-d789844eddd7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.158602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 942/1882] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:25.178348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:25.210161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0ed24-7a38-4e85-8968-ee437b1792bb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.218819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193946-a4e4-4be2-928e-b7b4f6217e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.219486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193946-a4e4-4be2-928e-b7b4f6217e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.219770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193946-a4e4-4be2-928e-b7b4f6217e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.220032 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193946-a4e4-4be2-928e-b7b4f6217e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.220773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193946-a4e4-4be2-928e-b7b4f6217e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.222351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193946-a4e4-4be2-928e-b7b4f6217e60 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.238954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81439050-eb73-4fc9-a04a-d789844eddd7 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:25.243125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 941/1883] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:25.247288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193946-a4e4-4be2-928e-b7b4f6217e60 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:25.255495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81439050-eb73-4fc9-a04a-d789844eddd7 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.259160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193946-a4e4-4be2-928e-b7b4f6217e60 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.262404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.263052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.263478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.263654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] PATH_INFO: `/v3/users/96f04be8ff2842bfad34cea858171b77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.264236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcf8f2-ab3e-4b7b-951d-a13fc24e1b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.265242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcf8f2-ab3e-4b7b-951d-a13fc24e1b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.265598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcf8f2-ab3e-4b7b-951d-a13fc24e1b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.265721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f04be8ff2842bfad34cea858171b77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.265968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcf8f2-ab3e-4b7b-951d-a13fc24e1b70 admin admin] PATH_INFO: `/v3/projects/077820bebf51491a87a466ab6403b86a/users/c53f2b3a14b9497e9fef8ef02535a96f/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.268232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcf8f2-ab3e-4b7b-951d-a13fc24e1b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=077820bebf51491a87a466ab6403b86a, user_id=c53f2b3a14b9497e9fef8ef02535a96f, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.269154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.271034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcf8f2-ab3e-4b7b-951d-a13fc24e1b70 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.283296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 942/1884] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:25 2022] PUT /identity/v3/projects/077820bebf51491a87a466ab6403b86a/users/c53f2b3a14b9497e9fef8ef02535a96f/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:25.293903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414dc1-e1d7-4988-9c9b-379167aef7f7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.294196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414dc1-e1d7-4988-9c9b-379167aef7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.295038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414dc1-e1d7-4988-9c9b-379167aef7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.325338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f04be8ff2842bfad34cea858171b77'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:25.326107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f04be8ff2842bfad34cea858171b77'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:25.327952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b6414dc1-e1d7-4988-9c9b-379167aef7f7 None None] MFA Rules not processed for user `c53f2b3a14b9497e9fef8ef02535a96f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:25.329337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b6414dc1-e1d7-4988-9c9b-379167aef7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:25.334128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f04be8ff2842bfad34cea858171b77'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:25.334734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f04be8ff2842bfad34cea858171b77'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:25.335938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:25.349495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e01d4c62-8e63-4198-a2a7-ba77117c5afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f04be8ff2842bfad34cea858171b77'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:25.362821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 943/1885] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:25 2022] DELETE /identity/v3/users/96f04be8ff2842bfad34cea858171b77 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:25.367329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323245a-0ae8-4772-bdba-2be76f680a3c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.367619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323245a-0ae8-4772-bdba-2be76f680a3c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.367887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323245a-0ae8-4772-bdba-2be76f680a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.387298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9323245a-0ae8-4772-bdba-2be76f680a3c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:25.388533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9323245a-0ae8-4772-bdba-2be76f680a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:25.403932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 943/1886] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:25.415900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6414dc1-e1d7-4988-9c9b-379167aef7f7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:25.417511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b6414dc1-e1d7-4988-9c9b-379167aef7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:25.449599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 944/1887] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:25.459526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323245a-0ae8-4772-bdba-2be76f680a3c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:25.461302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9323245a-0ae8-4772-bdba-2be76f680a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:25.477172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6414dc1-e1d7-4988-9c9b-379167aef7f7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.526028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323245a-0ae8-4772-bdba-2be76f680a3c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.528403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.529033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.529335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.529592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.530029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f4cb-eeac-4921-a48b-5bf5a5aaf52e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.530891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f4cb-eeac-4921-a48b-5bf5a5aaf52e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.531134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f4cb-eeac-4921-a48b-5bf5a5aaf52e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.531441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f4cb-eeac-4921-a48b-5bf5a5aaf52e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.543144 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535f4cb-eeac-4921-a48b-5bf5a5aaf52e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.543884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:25.544642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535f4cb-eeac-4921-a48b-5bf5a5aaf52e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.556392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 945/1888] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:25.605481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.607051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.618889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 944/1889] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:25.730056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f4cb-eeac-4921-a48b-5bf5a5aaf52e service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:25.740709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f4cb-eeac-4921-a48b-5bf5a5aaf52e service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.745824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.746512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.747279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.747279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.749324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:25.816785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.818235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.828348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 946/1890] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:25.863680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:25.865132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:25.883068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae870bfd-63af-4636-963b-b3024dc3d6b3 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:25.903711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:25.904459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:25.904643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:25.904910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] PATH_INFO: `/v3/projects/74abe8f121154e0796727b9d3a3444d6` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:25.907533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74abe8f121154e0796727b9d3a3444d6)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:25.909508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:25.971659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74abe8f121154e0796727b9d3a3444d6', 'request_id': 'req-73e9fde6-98fa-4343-a6f4-7966275a9ad4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:25.977763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74abe8f121154e0796727b9d3a3444d6', 'request_id': 'req-73e9fde6-98fa-4343-a6f4-7966275a9ad4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:25.984721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74abe8f121154e0796727b9d3a3444d6', 'request_id': 'req-73e9fde6-98fa-4343-a6f4-7966275a9ad4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:25.998412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] The token cache is being invalidate because project 74abe8f121154e0796727b9d3a3444d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:25.998824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.000138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.001680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 945/1891] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:25 2022] DELETE /identity/v3/projects/74abe8f121154e0796727b9d3a3444d6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:26.005829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:26.017547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ba4cf-b85b-4cbe-b226-4f1255c17eae service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:26.027334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:26.084809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:26.085493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:26.085736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:26.086006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] PATH_INFO: `/v3/users/4ce06e613d96442ea36d1c522c9da578` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:26.087186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce06e613d96442ea36d1c522c9da578)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:26.089224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:26.149706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce06e613d96442ea36d1c522c9da578'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.150498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce06e613d96442ea36d1c522c9da578'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.161281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce06e613d96442ea36d1c522c9da578'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.162200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce06e613d96442ea36d1c522c9da578'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.178429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe4f78f7-8dfd-4fe5-9f81-f8ba190febc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce06e613d96442ea36d1c522c9da578'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.198384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 947/1892] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:26 2022] DELETE /identity/v3/users/4ce06e613d96442ea36d1c522c9da578 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:26.615471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:26.616902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:26.630244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9fde6-98fa-4343-a6f4-7966275a9ad4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:26.692315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:26.693017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:26.693220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:26.693498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] PATH_INFO: `/v3/projects/a2ad200ec2ec4658aa077ef60b57ac2e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:26.694587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ad200ec2ec4658aa077ef60b57ac2e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:26.696506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:26.738072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:26.751810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ad200ec2ec4658aa077ef60b57ac2e', 'request_id': 'req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.759009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ad200ec2ec4658aa077ef60b57ac2e', 'request_id': 'req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.766211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ad200ec2ec4658aa077ef60b57ac2e', 'request_id': 'req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.781731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] The token cache is being invalidate because project a2ad200ec2ec4658aa077ef60b57ac2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:26.782179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.783098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6c0b58-31b2-4aae-b2cf-1a34d35634c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:26.785348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 946/1893] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:26 2022] DELETE /identity/v3/projects/a2ad200ec2ec4658aa077ef60b57ac2e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:26.792266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846ec8d-ffe0-429e-b08e-750e7b7a24eb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:26.792730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846ec8d-ffe0-429e-b08e-750e7b7a24eb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:26.793448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846ec8d-ffe0-429e-b08e-750e7b7a24eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:26.821519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f846ec8d-ffe0-429e-b08e-750e7b7a24eb None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:26.823264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f846ec8d-ffe0-429e-b08e-750e7b7a24eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:26.890641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 948/1894] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:26.898262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0d04-2cb0-488f-a0f2-33f054fbdc30 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:26.898701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0d04-2cb0-488f-a0f2-33f054fbdc30 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:26.898813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0d04-2cb0-488f-a0f2-33f054fbdc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:26.920081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b99a0d04-2cb0-488f-a0f2-33f054fbdc30 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:26.921333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b99a0d04-2cb0-488f-a0f2-33f054fbdc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:26.929928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 947/1895] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:26.934393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846ec8d-ffe0-429e-b08e-750e7b7a24eb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:26.945188 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846ec8d-ffe0-429e-b08e-750e7b7a24eb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:26.956129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:26.956788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:26.957061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:26.957329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:26.957981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:26.958280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:26.959969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:26.972818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 949/1896] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:26.977326 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a0d04-2cb0-488f-a0f2-33f054fbdc30 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:26.987025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a0d04-2cb0-488f-a0f2-33f054fbdc30 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:26.991657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6d51-2a4a-4c66-ad34-a39cf2459098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:26.992257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6d51-2a4a-4c66-ad34-a39cf2459098 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:26.992517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6d51-2a4a-4c66-ad34-a39cf2459098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:26.992797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6d51-2a4a-4c66-ad34-a39cf2459098 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:26.993433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6d51-2a4a-4c66-ad34-a39cf2459098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:26.995372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6d51-2a4a-4c66-ad34-a39cf2459098 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.026381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 948/1897] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:26 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:27.031304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:27.041691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2d5db-7a53-4117-a1b5-5e29a1277121 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:27.062059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:27.105332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d7bfd-b2e0-4d64-b0f9-62c19dd73147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:27.106139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d7bfd-b2e0-4d64-b0f9-62c19dd73147 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.106236 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d7bfd-b2e0-4d64-b0f9-62c19dd73147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.106494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d7bfd-b2e0-4d64-b0f9-62c19dd73147 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.107147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d7bfd-b2e0-4d64-b0f9-62c19dd73147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:27.109363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d7bfd-b2e0-4d64-b0f9-62c19dd73147 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.145121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 950/1898] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:27 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:27.150094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6d51-2a4a-4c66-ad34-a39cf2459098 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:27.163701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6d51-2a4a-4c66-ad34-a39cf2459098 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:27.169250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9701bf8-cb9a-4a94-a7ec-58011f9a7ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:27.169852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9701bf8-cb9a-4a94-a7ec-58011f9a7ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.170120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9701bf8-cb9a-4a94-a7ec-58011f9a7ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.170387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9701bf8-cb9a-4a94-a7ec-58011f9a7ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.171065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9701bf8-cb9a-4a94-a7ec-58011f9a7ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:27.172714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9701bf8-cb9a-4a94-a7ec-58011f9a7ef6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.183711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 949/1899] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:27.188208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d7bfd-b2e0-4d64-b0f9-62c19dd73147 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:27.199098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d7bfd-b2e0-4d64-b0f9-62c19dd73147 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:27.204410 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2263839-878f-4804-9dff-cc74b4059f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:27.205024 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2263839-878f-4804-9dff-cc74b4059f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.205311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2263839-878f-4804-9dff-cc74b4059f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.205584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2263839-878f-4804-9dff-cc74b4059f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.206223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2263839-878f-4804-9dff-cc74b4059f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:27.207880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2263839-878f-4804-9dff-cc74b4059f0d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.219151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 951/1900] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:27.223907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9701bf8-cb9a-4a94-a7ec-58011f9a7ef6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:27.241423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9701bf8-cb9a-4a94-a7ec-58011f9a7ef6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:27.248821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbdf8d-6d4f-4e5f-8064-9dba4c6db33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:27.249731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbdf8d-6d4f-4e5f-8064-9dba4c6db33c admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.250011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbdf8d-6d4f-4e5f-8064-9dba4c6db33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.250298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbdf8d-6d4f-4e5f-8064-9dba4c6db33c admin admin] PATH_INFO: `/v3/projects/081419a2f3c742feb86ada7a9e510d5a/users/8e15a8edb4164a498a1eb3b2105114c5/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.269393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbdf8d-6d4f-4e5f-8064-9dba4c6db33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=081419a2f3c742feb86ada7a9e510d5a, user_id=8e15a8edb4164a498a1eb3b2105114c5, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:27.272193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbdf8d-6d4f-4e5f-8064-9dba4c6db33c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.289696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 950/1901] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:27 2022] PUT /identity/v3/projects/081419a2f3c742feb86ada7a9e510d5a/users/8e15a8edb4164a498a1eb3b2105114c5/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:27.300940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbd4e-8b18-4dfe-9d8b-4ea205606aa7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.301513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbd4e-8b18-4dfe-9d8b-4ea205606aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.301854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbd4e-8b18-4dfe-9d8b-4ea205606aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.339366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a73fbd4e-8b18-4dfe-9d8b-4ea205606aa7 None None] MFA Rules not processed for user `8e15a8edb4164a498a1eb3b2105114c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:27.340863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a73fbd4e-8b18-4dfe-9d8b-4ea205606aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:27.489287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 952/1902] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:27.492814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbdf8d-6d4f-4e5f-8064-9dba4c6db33c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:27.638120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbdf8d-6d4f-4e5f-8064-9dba4c6db33c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:27.645632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3547bb-b785-4e98-a815-9ef53b42a93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:27.646500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3547bb-b785-4e98-a815-9ef53b42a93f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.648761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3547bb-b785-4e98-a815-9ef53b42a93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.649395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3547bb-b785-4e98-a815-9ef53b42a93f admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.650292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3547bb-b785-4e98-a815-9ef53b42a93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:27.652800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3547bb-b785-4e98-a815-9ef53b42a93f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.702365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 951/1903] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:27 2022] POST /identity/v3/projects => generated 393 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:27.704102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fbd4e-8b18-4dfe-9d8b-4ea205606aa7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:27.717949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fbd4e-8b18-4dfe-9d8b-4ea205606aa7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:27.798802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71abb5-5ab7-45f4-a02f-89780d6e454e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:27.800387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71abb5-5ab7-45f4-a02f-89780d6e454e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.800964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71abb5-5ab7-45f4-a02f-89780d6e454e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.801183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71abb5-5ab7-45f4-a02f-89780d6e454e admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.801805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71abb5-5ab7-45f4-a02f-89780d6e454e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:27.804293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71abb5-5ab7-45f4-a02f-89780d6e454e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.855894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 953/1904] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:27 2022] POST /identity/v3/users => generated 303 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:27.857097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3547bb-b785-4e98-a815-9ef53b42a93f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:27.869998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3547bb-b785-4e98-a815-9ef53b42a93f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:27.882993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36dec-d379-4f9a-b16c-aa85b6be1a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:27.883693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36dec-d379-4f9a-b16c-aa85b6be1a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.883978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36dec-d379-4f9a-b16c-aa85b6be1a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.884574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36dec-d379-4f9a-b16c-aa85b6be1a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.884899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36dec-d379-4f9a-b16c-aa85b6be1a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:27.886570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36dec-d379-4f9a-b16c-aa85b6be1a07 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.902768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 952/1905] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:27.907098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71abb5-5ab7-45f4-a02f-89780d6e454e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:27.919772 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71abb5-5ab7-45f4-a02f-89780d6e454e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:27.930487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d003315-cb7c-44e1-91ca-6618882f16d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:27.931347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d003315-cb7c-44e1-91ca-6618882f16d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:27.931787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d003315-cb7c-44e1-91ca-6618882f16d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:27.932181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d003315-cb7c-44e1-91ca-6618882f16d3 admin admin] PATH_INFO: `/v3/projects/90af433711dd4ed6b2c4e11bdd8453e4/users/41704b28ca3942b4b942b0c205a7e848/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:27.955333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d003315-cb7c-44e1-91ca-6618882f16d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90af433711dd4ed6b2c4e11bdd8453e4, user_id=41704b28ca3942b4b942b0c205a7e848, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:27.959023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d003315-cb7c-44e1-91ca-6618882f16d3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:27.970568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:27.991033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 954/1906] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:27 2022] PUT /identity/v3/projects/90af433711dd4ed6b2c4e11bdd8453e4/users/41704b28ca3942b4b942b0c205a7e848/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:27.995576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36dec-d379-4f9a-b16c-aa85b6be1a07 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:28.006797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36dec-d379-4f9a-b16c-aa85b6be1a07 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:28.044241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:28.088826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8526a7-33a5-4b25-9b39-7ece3805cdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:28.089446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8526a7-33a5-4b25-9b39-7ece3805cdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:28.089721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8526a7-33a5-4b25-9b39-7ece3805cdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:28.090038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8526a7-33a5-4b25-9b39-7ece3805cdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:28.090709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8526a7-33a5-4b25-9b39-7ece3805cdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:28.092349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8526a7-33a5-4b25-9b39-7ece3805cdd4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:28.104594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 953/1907] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:28.108531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d003315-cb7c-44e1-91ca-6618882f16d3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:28.118921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d003315-cb7c-44e1-91ca-6618882f16d3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:28.123665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:28.124504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:28.124787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:28.125038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] PATH_INFO: `/v3/projects/90af433711dd4ed6b2c4e11bdd8453e4/users/41704b28ca3942b4b942b0c205a7e848/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:28.127281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=90af433711dd4ed6b2c4e11bdd8453e4, user_id=41704b28ca3942b4b942b0c205a7e848, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:28.129933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:28.140302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 955/1908] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:28 2022] PUT /identity/v3/projects/90af433711dd4ed6b2c4e11bdd8453e4/users/41704b28ca3942b4b942b0c205a7e848/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:28.151698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29bbc6-8561-4faa-91b6-116e3ff3ceb1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:28.151980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29bbc6-8561-4faa-91b6-116e3ff3ceb1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:28.152250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29bbc6-8561-4faa-91b6-116e3ff3ceb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:28.180604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-aa29bbc6-8561-4faa-91b6-116e3ff3ceb1 None None] MFA Rules not processed for user `41704b28ca3942b4b942b0c205a7e848`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:28.181804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aa29bbc6-8561-4faa-91b6-116e3ff3ceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:28.257805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 954/1909] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:28.270778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:28.272589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:28.329290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02e64a-49b3-4f4e-b7b8-a38ddba6c09b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:28.333635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:28.334243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:28.334517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:28.334781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:28.344995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:28.346412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:28.357840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 956/1910] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:28 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:28.407637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29bbc6-8561-4faa-91b6-116e3ff3ceb1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:28.409236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aa29bbc6-8561-4faa-91b6-116e3ff3ceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:28.421858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29bbc6-8561-4faa-91b6-116e3ff3ceb1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:28.426542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd16b5c-56b0-41e0-835a-5e0ba47b918e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:28.427154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd16b5c-56b0-41e0-835a-5e0ba47b918e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:28.427469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd16b5c-56b0-41e0-835a-5e0ba47b918e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:28.427740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd16b5c-56b0-41e0-835a-5e0ba47b918e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:28.486515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd16b5c-56b0-41e0-835a-5e0ba47b918e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:28.488071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd16b5c-56b0-41e0-835a-5e0ba47b918e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:28.498240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 955/1911] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:31.921097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:31.922608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:31.994120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7fbfd-8912-47d0-a777-81b2f7c6e5a0 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:31.999062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:31.999756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:32.000029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:32.000379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] PATH_INFO: `/v3/users/c5ac3ea0e7214ba0b47fb170e56e6db9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:32.001533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ac3ea0e7214ba0b47fb170e56e6db9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:32.003589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:32.056897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ac3ea0e7214ba0b47fb170e56e6db9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.057974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ac3ea0e7214ba0b47fb170e56e6db9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.068507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ac3ea0e7214ba0b47fb170e56e6db9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.069120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ac3ea0e7214ba0b47fb170e56e6db9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.083759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ac3ea0e7214ba0b47fb170e56e6db9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.101151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 957/1912] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:31 2022] DELETE /identity/v3/users/c5ac3ea0e7214ba0b47fb170e56e6db9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:32.103306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745b1c-0165-4230-b0a9-7aa58158f1a2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:32.103880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745b1c-0165-4230-b0a9-7aa58158f1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:32.104116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745b1c-0165-4230-b0a9-7aa58158f1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:32.124676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d5745b1c-0165-4230-b0a9-7aa58158f1a2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:32.125996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d5745b1c-0165-4230-b0a9-7aa58158f1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:32.200178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:32.219515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 956/1913] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:32.229460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:32.231482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:32.238076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:32.303260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861f511-184e-459a-bac5-479bbdf774a2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:32.307481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d5c15-2e72-4d94-a3f7-de65482aed24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:32.308494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5c15-2e72-4d94-a3f7-de65482aed24 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:32.308759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5c15-2e72-4d94-a3f7-de65482aed24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:32.309005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5c15-2e72-4d94-a3f7-de65482aed24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:32.319318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d5c15-2e72-4d94-a3f7-de65482aed24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:32.320768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d5c15-2e72-4d94-a3f7-de65482aed24 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:32.330327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 958/1914] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:32.648850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5745b1c-0165-4230-b0a9-7aa58158f1a2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:32.654039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d5745b1c-0165-4230-b0a9-7aa58158f1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:32.711274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5745b1c-0165-4230-b0a9-7aa58158f1a2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:32.721778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:32.723697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:32.724578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:32.725337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] PATH_INFO: `/v3/projects/83dae260602c4a0288449110410a4337` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:32.728378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=83dae260602c4a0288449110410a4337)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:32.733833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:32.865459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83dae260602c4a0288449110410a4337', 'request_id': 'req-469caa24-c1f4-4992-beb5-7f0528a1b3eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.872194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83dae260602c4a0288449110410a4337', 'request_id': 'req-469caa24-c1f4-4992-beb5-7f0528a1b3eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.882917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83dae260602c4a0288449110410a4337', 'request_id': 'req-469caa24-c1f4-4992-beb5-7f0528a1b3eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.909933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] The token cache is being invalidate because project 83dae260602c4a0288449110410a4337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:32.910275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.911024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:32.912883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 957/1915] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:32 2022] DELETE /identity/v3/projects/83dae260602c4a0288449110410a4337 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:32.917208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d5c15-2e72-4d94-a3f7-de65482aed24 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:32.958018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d5c15-2e72-4d94-a3f7-de65482aed24 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:33.119277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:33.120372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:33.121827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:33.122283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] PATH_INFO: `/v3/users/c53f2b3a14b9497e9fef8ef02535a96f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:33.125559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c53f2b3a14b9497e9fef8ef02535a96f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:33.128299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:33.207543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c53f2b3a14b9497e9fef8ef02535a96f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.208397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c53f2b3a14b9497e9fef8ef02535a96f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.222502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c53f2b3a14b9497e9fef8ef02535a96f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.223185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c53f2b3a14b9497e9fef8ef02535a96f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.241898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:33.243526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fb76ae0a-98d9-426f-ac5b-2478a885a8bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c53f2b3a14b9497e9fef8ef02535a96f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.244632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:33.259867 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 959/1916] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:32 2022] DELETE /identity/v3/users/c53f2b3a14b9497e9fef8ef02535a96f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:33.267834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469caa24-c1f4-4992-beb5-7f0528a1b3eb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:33.419151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:33.419827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:33.420115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:33.420488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] PATH_INFO: `/v3/users/8e15a8edb4164a498a1eb3b2105114c5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:33.444790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e15a8edb4164a498a1eb3b2105114c5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:33.446954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:33.541179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e15a8edb4164a498a1eb3b2105114c5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.542151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e15a8edb4164a498a1eb3b2105114c5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.560466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e15a8edb4164a498a1eb3b2105114c5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.571954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e15a8edb4164a498a1eb3b2105114c5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.585203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e15a8edb4164a498a1eb3b2105114c5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.587646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:33.598610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 958/1917] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:33 2022] DELETE /identity/v3/users/8e15a8edb4164a498a1eb3b2105114c5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:33.601513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f8c2e-9217-4f7d-8e31-ec8c8f78189d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:33.602027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f8c2e-9217-4f7d-8e31-ec8c8f78189d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:33.603490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f8c2e-9217-4f7d-8e31-ec8c8f78189d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:33.630717 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-531f8c2e-9217-4f7d-8e31-ec8c8f78189d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:33.632287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-531f8c2e-9217-4f7d-8e31-ec8c8f78189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:33.658915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:33.719367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 960/1918] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:33.735343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:33.737247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:33.805135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0dbb4-066b-432d-982c-ab8c0eb76ea5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:33.813185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817e734-2613-4320-8786-ff0db69658ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:33.813866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e734-2613-4320-8786-ff0db69658ad service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:33.814140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e734-2613-4320-8786-ff0db69658ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:33.814430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817e734-2613-4320-8786-ff0db69658ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:33.829381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817e734-2613-4320-8786-ff0db69658ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:33.830222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f8c2e-9217-4f7d-8e31-ec8c8f78189d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:33.831389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817e734-2613-4320-8786-ff0db69658ad service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:33.831879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-531f8c2e-9217-4f7d-8e31-ec8c8f78189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:33.848282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f8c2e-9217-4f7d-8e31-ec8c8f78189d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:33.854372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:33.855178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:33.855547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:33.856036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] PATH_INFO: `/v3/projects/077820bebf51491a87a466ab6403b86a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:33.857571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] RBAC: Authorizing `identity:delete_project(project_id=077820bebf51491a87a466ab6403b86a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:33.859481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 959/1919] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:33.860019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:33.920703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '077820bebf51491a87a466ab6403b86a', 'request_id': 'req-f005b450-db23-4979-8a4c-a45a4c83f3de'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.932982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '077820bebf51491a87a466ab6403b86a', 'request_id': 'req-f005b450-db23-4979-8a4c-a45a4c83f3de'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.958478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '077820bebf51491a87a466ab6403b86a', 'request_id': 'req-f005b450-db23-4979-8a4c-a45a4c83f3de'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.987195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] The token cache is being invalidate because project 077820bebf51491a87a466ab6403b86a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:33.987634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.988455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f005b450-db23-4979-8a4c-a45a4c83f3de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:33.997266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 961/1920] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:33 2022] DELETE /identity/v3/projects/077820bebf51491a87a466ab6403b86a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:33.998969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ba8ea-546c-4158-b550-f21a486547ec None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:33.999273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ba8ea-546c-4158-b550-f21a486547ec None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:33.999744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ba8ea-546c-4158-b550-f21a486547ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.025858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-809ba8ea-546c-4158-b550-f21a486547ec None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:34.028171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-809ba8ea-546c-4158-b550-f21a486547ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:34.128109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 960/1921] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:34.139864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5af1b-77e8-422c-998d-9d5750b7bae4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.140364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5af1b-77e8-422c-998d-9d5750b7bae4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.141148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5af1b-77e8-422c-998d-9d5750b7bae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.192226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-58d5af1b-77e8-422c-998d-9d5750b7bae4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:34.193455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-58d5af1b-77e8-422c-998d-9d5750b7bae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:34.211140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 962/1922] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:34.215930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ba8ea-546c-4158-b550-f21a486547ec None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:34.230999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ba8ea-546c-4158-b550-f21a486547ec None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:34.244794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:34.246101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.246485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.246907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.247714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:34.248171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:34.252212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:34.270000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 961/1923] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:34.283647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5af1b-77e8-422c-998d-9d5750b7bae4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:34.294868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5af1b-77e8-422c-998d-9d5750b7bae4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:34.303928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595538-ef8a-46c5-b7ef-3d56a443fc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:34.304641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595538-ef8a-46c5-b7ef-3d56a443fc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.304990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595538-ef8a-46c5-b7ef-3d56a443fc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.305327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595538-ef8a-46c5-b7ef-3d56a443fc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.306039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07595538-ef8a-46c5-b7ef-3d56a443fc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:34.308666 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07595538-ef8a-46c5-b7ef-3d56a443fc3a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:34.367628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:34.369626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:34.370539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 963/1924] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:34 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:34.374702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595538-ef8a-46c5-b7ef-3d56a443fc3a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:34.389547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528769-9168-4552-a833-d2836a31bc85 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:34.519904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595538-ef8a-46c5-b7ef-3d56a443fc3a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:34.526369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:34.527370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.527628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.527943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] PATH_INFO: `/v3/projects/081419a2f3c742feb86ada7a9e510d5a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.530454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=081419a2f3c742feb86ada7a9e510d5a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:34.533416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:34.536874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b65da-ac5a-4f57-9030-6e0f75544297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:34.537508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b65da-ac5a-4f57-9030-6e0f75544297 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.537786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b65da-ac5a-4f57-9030-6e0f75544297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.538055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b65da-ac5a-4f57-9030-6e0f75544297 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.539121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b65da-ac5a-4f57-9030-6e0f75544297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:34.542021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b65da-ac5a-4f57-9030-6e0f75544297 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:34.597849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 964/1925] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:34 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:34.602811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b65da-ac5a-4f57-9030-6e0f75544297 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:34.603138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:34.616334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b65da-ac5a-4f57-9030-6e0f75544297 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:34.623635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:34.626927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '081419a2f3c742feb86ada7a9e510d5a', 'request_id': 'req-6fda75bc-dffb-4419-b4b7-3ef918a73f43'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.638962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '081419a2f3c742feb86ada7a9e510d5a', 'request_id': 'req-6fda75bc-dffb-4419-b4b7-3ef918a73f43'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.648604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '081419a2f3c742feb86ada7a9e510d5a', 'request_id': 'req-6fda75bc-dffb-4419-b4b7-3ef918a73f43'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.669319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] The token cache is being invalidate because project 081419a2f3c742feb86ada7a9e510d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:34.669648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.671460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.674125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 962/1926] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:34 2022] DELETE /identity/v3/projects/081419a2f3c742feb86ada7a9e510d5a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:34.679040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:34.680848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:34.700154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:34.701118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.701673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.702122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.703034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:34.706129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:34.722616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 965/1927] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:34.723902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:34.726324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:34.740940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b3819-7102-42d5-b5c9-90eed644900d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:34.747005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2c167-bc8c-4c4c-a5c6-a2a3f62acd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:34.749508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c167-bc8c-4c4c-a5c6-a2a3f62acd39 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.750518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c167-bc8c-4c4c-a5c6-a2a3f62acd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.750760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c167-bc8c-4c4c-a5c6-a2a3f62acd39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.751702 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2c167-bc8c-4c4c-a5c6-a2a3f62acd39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:34.753487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2c167-bc8c-4c4c-a5c6-a2a3f62acd39 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:34.764547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda75bc-dffb-4419-b4b7-3ef918a73f43 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:34.768102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 966/1928] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:34.770028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:34.770671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.771154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.771487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] PATH_INFO: `/v3/users/41704b28ca3942b4b942b0c205a7e848` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.772220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2c167-bc8c-4c4c-a5c6-a2a3f62acd39 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:34.773168 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41704b28ca3942b4b942b0c205a7e848)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:34.775952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:34.784733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2c167-bc8c-4c4c-a5c6-a2a3f62acd39 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:34.794122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d77229-56cd-4ca7-a49c-98d3ed55cc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:34.794848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77229-56cd-4ca7-a49c-98d3ed55cc54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.795134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77229-56cd-4ca7-a49c-98d3ed55cc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.797425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77229-56cd-4ca7-a49c-98d3ed55cc54 admin admin] PATH_INFO: `/v3/projects/0d1b440ef8644dfc82577c155ad743da/users/9f7907cabdd14fdd81d6bddccb471bd4/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.822225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d77229-56cd-4ca7-a49c-98d3ed55cc54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1b440ef8644dfc82577c155ad743da, user_id=9f7907cabdd14fdd81d6bddccb471bd4, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:34.826455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d77229-56cd-4ca7-a49c-98d3ed55cc54 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:34.845601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 967/1929] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:34 2022] PUT /identity/v3/projects/0d1b440ef8644dfc82577c155ad743da/users/9f7907cabdd14fdd81d6bddccb471bd4/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:34.852993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41704b28ca3942b4b942b0c205a7e848'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.853673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41704b28ca3942b4b942b0c205a7e848'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.859039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f953c5-445c-446d-b157-d8f3cc851eba None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:34.862072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f953c5-445c-446d-b157-d8f3cc851eba None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:34.862466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f953c5-445c-446d-b157-d8f3cc851eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:34.900040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41704b28ca3942b4b942b0c205a7e848'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.918108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41704b28ca3942b4b942b0c205a7e848'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.937821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41704b28ca3942b4b942b0c205a7e848'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:34.950480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 963/1930] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:34 2022] DELETE /identity/v3/users/41704b28ca3942b4b942b0c205a7e848 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:34.965781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-54f953c5-445c-446d-b157-d8f3cc851eba None None] MFA Rules not processed for user `9f7907cabdd14fdd81d6bddccb471bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:34.967499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-54f953c5-445c-446d-b157-d8f3cc851eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:35.102039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 968/1931] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:35.115156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:35.116817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:35.178507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:35.236519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb286-e8da-4071-bff5-7fe51f7d55b9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:35.241002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6259916-6a30-4356-8a6e-c9ca4072d598 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:35.241746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259916-6a30-4356-8a6e-c9ca4072d598 service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:35.242128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259916-6a30-4356-8a6e-c9ca4072d598 service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:35.242501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259916-6a30-4356-8a6e-c9ca4072d598 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:35.252566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6259916-6a30-4356-8a6e-c9ca4072d598 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:35.254560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6259916-6a30-4356-8a6e-c9ca4072d598 service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:35.265373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 964/1932] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:35.373025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f953c5-445c-446d-b157-d8f3cc851eba None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:35.375062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-54f953c5-445c-446d-b157-d8f3cc851eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:35.495313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f953c5-445c-446d-b157-d8f3cc851eba None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:35.504540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:35.505537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:35.505851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:35.506476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] PATH_INFO: `/v3/projects/90af433711dd4ed6b2c4e11bdd8453e4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:35.507322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=90af433711dd4ed6b2c4e11bdd8453e4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:35.509508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:35.511447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:35.621195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90af433711dd4ed6b2c4e11bdd8453e4', 'request_id': 'req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:35.629773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90af433711dd4ed6b2c4e11bdd8453e4', 'request_id': 'req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:35.649258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90af433711dd4ed6b2c4e11bdd8453e4', 'request_id': 'req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:35.663041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] The token cache is being invalidate because project 90af433711dd4ed6b2c4e11bdd8453e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:35.663301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:35.663982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e08b2cf4-0c33-4166-a73d-65b7114ce2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:35.665831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 969/1933] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:35 2022] DELETE /identity/v3/projects/90af433711dd4ed6b2c4e11bdd8453e4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:35.674072 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14284506-5492-4786-a5f8-8f5c758f67ef None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:35.674428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14284506-5492-4786-a5f8-8f5c758f67ef None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:35.674803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14284506-5492-4786-a5f8-8f5c758f67ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:35.699990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-14284506-5492-4786-a5f8-8f5c758f67ef None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:35.701939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14284506-5492-4786-a5f8-8f5c758f67ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:35.788762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 965/1934] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:35.790460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad5343-4ec5-4d6a-a2d3-861ba1ddac82 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:35.790797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad5343-4ec5-4d6a-a2d3-861ba1ddac82 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:35.791018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad5343-4ec5-4d6a-a2d3-861ba1ddac82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:35.836258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6aad5343-4ec5-4d6a-a2d3-861ba1ddac82 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:35.837569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6aad5343-4ec5-4d6a-a2d3-861ba1ddac82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:35.865325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 970/1935] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:35.870449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14284506-5492-4786-a5f8-8f5c758f67ef None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:35.884566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14284506-5492-4786-a5f8-8f5c758f67ef None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:35.900543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:35.901483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:35.902076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:35.902502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:35.903571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:35.904359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:35.907427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:35.927296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 966/1936] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:35.930717 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad5343-4ec5-4d6a-a2d3-861ba1ddac82 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:35.941049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad5343-4ec5-4d6a-a2d3-861ba1ddac82 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:35.955514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053a9b1-1757-444d-8cec-b67bf27739f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:35.956390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053a9b1-1757-444d-8cec-b67bf27739f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:35.956721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053a9b1-1757-444d-8cec-b67bf27739f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:35.956994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053a9b1-1757-444d-8cec-b67bf27739f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:35.959427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053a9b1-1757-444d-8cec-b67bf27739f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:35.964640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053a9b1-1757-444d-8cec-b67bf27739f9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:36.023151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 971/1937] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:35 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:36.034510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:36.051986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d3d5b-5b40-4094-b0d8-b6c7c0d6d2d6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:36.195971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6304818-066d-4ae8-9e62-105a3b647c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:36.196793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6304818-066d-4ae8-9e62-105a3b647c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:36.197188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6304818-066d-4ae8-9e62-105a3b647c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:36.197521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6304818-066d-4ae8-9e62-105a3b647c69 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:36.198254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6304818-066d-4ae8-9e62-105a3b647c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:36.200364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6304818-066d-4ae8-9e62-105a3b647c69 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:36.240035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 967/1938] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:36 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:36.244751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053a9b1-1757-444d-8cec-b67bf27739f9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:36.264067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053a9b1-1757-444d-8cec-b67bf27739f9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:36.279776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b5d0c-d4cd-4db4-b303-e6ece269abbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:36.280776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b5d0c-d4cd-4db4-b303-e6ece269abbe admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:36.281316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b5d0c-d4cd-4db4-b303-e6ece269abbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:36.281642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b5d0c-d4cd-4db4-b303-e6ece269abbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:36.282457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b5d0c-d4cd-4db4-b303-e6ece269abbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:36.284691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b5d0c-d4cd-4db4-b303-e6ece269abbe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:36.318874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 972/1939] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:36.320312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6304818-066d-4ae8-9e62-105a3b647c69 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:36.336106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6304818-066d-4ae8-9e62-105a3b647c69 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:36.365813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3ed57-8351-47a0-aaee-600717184aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:36.366883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3ed57-8351-47a0-aaee-600717184aff admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:36.372864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3ed57-8351-47a0-aaee-600717184aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:36.374648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3ed57-8351-47a0-aaee-600717184aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:36.377173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3ed57-8351-47a0-aaee-600717184aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:36.385074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3ed57-8351-47a0-aaee-600717184aff admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:36.438911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 968/1940] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:36.444151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b5d0c-d4cd-4db4-b303-e6ece269abbe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:36.461224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b5d0c-d4cd-4db4-b303-e6ece269abbe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:36.485567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:36.486451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:36.486771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:36.487127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] PATH_INFO: `/v3/projects/38e8143075444077bff9c942c36b5ca2/users/57ed00a25b6b4d6188f0771456570c6b/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:36.559420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e8143075444077bff9c942c36b5ca2, user_id=57ed00a25b6b4d6188f0771456570c6b, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:36.559420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:36.603824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 973/1941] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:36 2022] PUT /identity/v3/projects/38e8143075444077bff9c942c36b5ca2/users/57ed00a25b6b4d6188f0771456570c6b/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:36.613277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458633f-4935-4adb-8202-7808cd6b6171 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:36.613825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458633f-4935-4adb-8202-7808cd6b6171 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:36.614246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458633f-4935-4adb-8202-7808cd6b6171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:36.670934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d458633f-4935-4adb-8202-7808cd6b6171 None None] MFA Rules not processed for user `57ed00a25b6b4d6188f0771456570c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:36.678993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d458633f-4935-4adb-8202-7808cd6b6171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:36.712583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:36.777064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 969/1942] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:36.790551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:36.793811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:36.880593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad879d2-1018-4b4a-86fd-fef4c752f73f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:36.884536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1269a-7c25-463a-90be-a3ba2cb0042d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:36.885163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1269a-7c25-463a-90be-a3ba2cb0042d service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:36.885444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1269a-7c25-463a-90be-a3ba2cb0042d service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:36.885768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1269a-7c25-463a-90be-a3ba2cb0042d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:36.902520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1269a-7c25-463a-90be-a3ba2cb0042d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:36.904043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1269a-7c25-463a-90be-a3ba2cb0042d service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:36.915333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 974/1943] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:36.940341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458633f-4935-4adb-8202-7808cd6b6171 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:37.019195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458633f-4935-4adb-8202-7808cd6b6171 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:37.024635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:37.025340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:37.025613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:37.025879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] PATH_INFO: `/v3/users/57ed00a25b6b4d6188f0771456570c6b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:37.028058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ed00a25b6b4d6188f0771456570c6b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:37.031594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:37.105877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ed00a25b6b4d6188f0771456570c6b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:37.106825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ed00a25b6b4d6188f0771456570c6b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:37.137676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ed00a25b6b4d6188f0771456570c6b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:37.152816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ed00a25b6b4d6188f0771456570c6b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:37.163268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-71725f9c-fd6e-4570-81cb-747f853723bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ed00a25b6b4d6188f0771456570c6b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:37.178054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 970/1944] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:36 2022] DELETE /identity/v3/users/57ed00a25b6b4d6188f0771456570c6b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:37.185159 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4f66-6074-4b26-9844-b2670ff69e7e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:37.185711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4f66-6074-4b26-9844-b2670ff69e7e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:37.185711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4f66-6074-4b26-9844-b2670ff69e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:37.210926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0ab3f-29be-4479-aca0-31036020aba9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:37.211531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0ab3f-29be-4479-aca0-31036020aba9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:37.211944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0ab3f-29be-4479-aca0-31036020aba9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:37.213500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 971/1945] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:19:37.216444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-aa5d4f66-6074-4b26-9844-b2670ff69e7e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:37.218238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aa5d4f66-6074-4b26-9844-b2670ff69e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:37.228450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02529-7441-4dde-85dd-5ca17323fff0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:37.228758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02529-7441-4dde-85dd-5ca17323fff0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:37.229003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02529-7441-4dde-85dd-5ca17323fff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:37.249366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d9f02529-7441-4dde-85dd-5ca17323fff0 None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:37.250459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:37.251827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9f02529-7441-4dde-85dd-5ca17323fff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:37.327819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:37.351324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 975/1946] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:37.386984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 972/1947] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:19:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:37.428650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d4f66-6074-4b26-9844-b2670ff69e7e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:37.431055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aa5d4f66-6074-4b26-9844-b2670ff69e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:37.468235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a437-4d64-4d72-a725-7660fe005ef6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:37.468660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a437-4d64-4d72-a725-7660fe005ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:37.468941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a437-4d64-4d72-a725-7660fe005ef6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:37.470680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 973/1948] 10.222.0.27 () {54 vars in 961 bytes} [Wed Aug 31 12:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:19:37.478537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6cfa-afc2-463f-9ece-7d39d2abd5da None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:37.478822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6cfa-afc2-463f-9ece-7d39d2abd5da None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:37.479093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6cfa-afc2-463f-9ece-7d39d2abd5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:37.527679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ba1a6cfa-afc2-463f-9ece-7d39d2abd5da None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:37.534881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ba1a6cfa-afc2-463f-9ece-7d39d2abd5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:37.569442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 974/1949] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Aug 31 12:19:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:37.577263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a6cfa-afc2-463f-9ece-7d39d2abd5da None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:37.594022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d4f66-6074-4b26-9844-b2670ff69e7e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:37.599122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a6cfa-afc2-463f-9ece-7d39d2abd5da None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:37.603517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:37.604212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:37.604664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:37.604945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:37.610896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfcf84-13dc-4959-989a-9e2be34cbc75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:37.611794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfcf84-13dc-4959-989a-9e2be34cbc75 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:37.612269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfcf84-13dc-4959-989a-9e2be34cbc75 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:37.612747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfcf84-13dc-4959-989a-9e2be34cbc75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:37.624315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfcf84-13dc-4959-989a-9e2be34cbc75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:37.626549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfcf84-13dc-4959-989a-9e2be34cbc75 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:37.627902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:37.629368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:37.646463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 976/1950] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:37.651881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 975/1951] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:38.425096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:38.426424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:38.442254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecca305-d235-416f-a6c2-81de9d4ca405 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:38.448392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:38.449334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.449622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.449966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] PATH_INFO: `/v3/projects/38e8143075444077bff9c942c36b5ca2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.451087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e8143075444077bff9c942c36b5ca2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:38.453174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:38.514756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e8143075444077bff9c942c36b5ca2', 'request_id': 'req-eb0cb33e-6d93-4dc5-9454-08d77815c442'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:38.521769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e8143075444077bff9c942c36b5ca2', 'request_id': 'req-eb0cb33e-6d93-4dc5-9454-08d77815c442'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:38.532329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e8143075444077bff9c942c36b5ca2', 'request_id': 'req-eb0cb33e-6d93-4dc5-9454-08d77815c442'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:38.547427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] The token cache is being invalidate because project 38e8143075444077bff9c942c36b5ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:38.547891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:38.548910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-eb0cb33e-6d93-4dc5-9454-08d77815c442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:38.550528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 977/1952] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:38 2022] DELETE /identity/v3/projects/38e8143075444077bff9c942c36b5ca2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:38.555563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89fbc8-cca3-4388-8107-701d93f42230 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.556059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89fbc8-cca3-4388-8107-701d93f42230 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.556147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89fbc8-cca3-4388-8107-701d93f42230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.576991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6d89fbc8-cca3-4388-8107-701d93f42230 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:38.578206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6d89fbc8-cca3-4388-8107-701d93f42230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:38.651536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 976/1953] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:38.659394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8517-610d-4f54-a759-d32307201c6d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.659661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8517-610d-4f54-a759-d32307201c6d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.660120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8517-610d-4f54-a759-d32307201c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.680847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-981a8517-610d-4f54-a759-d32307201c6d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:38.682116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-981a8517-610d-4f54-a759-d32307201c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:38.699792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89fbc8-cca3-4388-8107-701d93f42230 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:38.700210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 978/1954] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:38.711280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89fbc8-cca3-4388-8107-701d93f42230 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:38.717102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:38.717729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.718013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.718276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.718926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:38.719254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:38.720896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:38.732869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 977/1955] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:38.737350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a8517-610d-4f54-a759-d32307201c6d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:38.746613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a8517-610d-4f54-a759-d32307201c6d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:38.751950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef812bd-397f-4404-bfa9-e94fab765de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:38.752549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef812bd-397f-4404-bfa9-e94fab765de5 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.752828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef812bd-397f-4404-bfa9-e94fab765de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.753099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef812bd-397f-4404-bfa9-e94fab765de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.753761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef812bd-397f-4404-bfa9-e94fab765de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:38.755780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef812bd-397f-4404-bfa9-e94fab765de5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:38.789588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 979/1956] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:38 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:38.790921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:38.801597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5b514-f04f-42af-a163-c321f1462aef admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:38.871947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cff89-3422-4f37-bed3-86c366680fb9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.872197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c2b55-a901-44cf-83ab-ba0c7e6b2d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:38.872638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cff89-3422-4f37-bed3-86c366680fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.872949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2b55-a901-44cf-83ab-ba0c7e6b2d3b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.873113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cff89-3422-4f37-bed3-86c366680fb9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.873441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2b55-a901-44cf-83ab-ba0c7e6b2d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.873876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2b55-a901-44cf-83ab-ba0c7e6b2d3b admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.874592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 980/1957] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:19:38.874960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c2b55-a901-44cf-83ab-ba0c7e6b2d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:38.877124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c2b55-a901-44cf-83ab-ba0c7e6b2d3b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:38.891773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9fd2e-d06a-4e65-8987-b2ce1b4ad213 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.892143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9fd2e-d06a-4e65-8987-b2ce1b4ad213 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.892483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9fd2e-d06a-4e65-8987-b2ce1b4ad213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.913656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e0a9fd2e-d06a-4e65-8987-b2ce1b4ad213 None None] MFA Rules not processed for user `56361f445d894e73bae29764ae8b2b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:38.914913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e0a9fd2e-d06a-4e65-8987-b2ce1b4ad213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:38.916885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 978/1958] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:38 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:38.922400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c2b55-a901-44cf-83ab-ba0c7e6b2d3b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:38.936684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c2b55-a901-44cf-83ab-ba0c7e6b2d3b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:38.941994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558ddb7-1c5c-41d6-ad91-bfdd9c348ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:38.942212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:38.942540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ddb7-1c5c-41d6-ad91-bfdd9c348ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.942823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ddb7-1c5c-41d6-ad91-bfdd9c348ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.943093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ddb7-1c5c-41d6-ad91-bfdd9c348ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.943822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558ddb7-1c5c-41d6-ad91-bfdd9c348ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:38.945797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558ddb7-1c5c-41d6-ad91-bfdd9c348ecc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:38.959908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 979/1959] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:38.964435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558ddb7-1c5c-41d6-ad91-bfdd9c348ecc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:38.986147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558ddb7-1c5c-41d6-ad91-bfdd9c348ecc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:38.991962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:38.992576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:38.992834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:38.993109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:38.993828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:38.995515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:39.010588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 980/1960] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:39.010588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:39.010943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:39.023846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48eab2-d4a0-46bd-a8da-8d83654849e9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:39.028737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:39.028995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 981/1961] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:19:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:39.029676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:39.029960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:39.030066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9fd2e-d06a-4e65-8987-b2ce1b4ad213 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:39.030236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] PATH_INFO: `/v3/users/befc6c8959484a18b0a0a4f01a6bb44d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:39.031704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] RBAC: Authorizing `identity:delete_user(user_id=befc6c8959484a18b0a0a4f01a6bb44d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:39.033749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:39.039310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:39.045953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9fd2e-d06a-4e65-8987-b2ce1b4ad213 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:39.051319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:39.051995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:39.052270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:39.052546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] PATH_INFO: `/v3/projects/b804356a5b6a45eb9789286a23b69e0d/users/942c167ca1ba466cbf2ded0f4ab6b2e2/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:39.070646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b804356a5b6a45eb9789286a23b69e0d, user_id=942c167ca1ba466cbf2ded0f4ab6b2e2, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:39.074604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:39.090827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 982/1962] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:39 2022] PUT /identity/v3/projects/b804356a5b6a45eb9789286a23b69e0d/users/942c167ca1ba466cbf2ded0f4ab6b2e2/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:39.093219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'befc6c8959484a18b0a0a4f01a6bb44d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.095818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'befc6c8959484a18b0a0a4f01a6bb44d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.096545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:39.098863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:39.110164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'befc6c8959484a18b0a0a4f01a6bb44d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.121172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'befc6c8959484a18b0a0a4f01a6bb44d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.130050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b930a2f3-d303-4caa-af48-dea70b0c714e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'befc6c8959484a18b0a0a4f01a6bb44d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.137874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 981/1963] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:39 2022] DELETE /identity/v3/users/befc6c8959484a18b0a0a4f01a6bb44d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:39.139899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160afdba-dfea-4f82-8eb4-1e9102df6e1d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:39.140181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160afdba-dfea-4f82-8eb4-1e9102df6e1d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:39.140458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160afdba-dfea-4f82-8eb4-1e9102df6e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:39.178257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-160afdba-dfea-4f82-8eb4-1e9102df6e1d None None] MFA Rules not processed for user `942c167ca1ba466cbf2ded0f4ab6b2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:39.179785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-160afdba-dfea-4f82-8eb4-1e9102df6e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:39.184483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeefd2-f63e-4953-a6a7-4a9f7bdc34d3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:39.274386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 982/1964] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:39.278928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:39.279725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:39.280017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:39.280288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:39.282078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:39.285961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160afdba-dfea-4f82-8eb4-1e9102df6e1d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:39.287745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-160afdba-dfea-4f82-8eb4-1e9102df6e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:39.299270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:39.301608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:39.316251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 983/1965] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:39.360722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:39.367519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:39.380930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160afdba-dfea-4f82-8eb4-1e9102df6e1d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:39.384820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25331-de35-4c05-bfd0-d530e71a300a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:39.386483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25331-de35-4c05-bfd0-d530e71a300a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:39.386778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25331-de35-4c05-bfd0-d530e71a300a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:39.387050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25331-de35-4c05-bfd0-d530e71a300a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:39.396366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d25331-de35-4c05-bfd0-d530e71a300a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:39.397819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d25331-de35-4c05-bfd0-d530e71a300a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:39.407285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 983/1966] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:39.408393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:39.463376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc56ee-66ad-46ef-9796-f72fde811625 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:39.468303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:39.468943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:39.469216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:39.469517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] PATH_INFO: `/v3/projects/c15bed7fec3c4f81a73182567caa30bd` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:39.470643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15bed7fec3c4f81a73182567caa30bd)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:39.472592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:39.526271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15bed7fec3c4f81a73182567caa30bd', 'request_id': 'req-e32f4bcf-aef1-47f2-8783-0793f4a34462'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.546870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15bed7fec3c4f81a73182567caa30bd', 'request_id': 'req-e32f4bcf-aef1-47f2-8783-0793f4a34462'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.557999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15bed7fec3c4f81a73182567caa30bd', 'request_id': 'req-e32f4bcf-aef1-47f2-8783-0793f4a34462'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.571648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] The token cache is being invalidate because project c15bed7fec3c4f81a73182567caa30bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:39.572183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.573016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.576133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 984/1967] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:39 2022] DELETE /identity/v3/projects/c15bed7fec3c4f81a73182567caa30bd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:39.580765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25331-de35-4c05-bfd0-d530e71a300a service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:39.594867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25331-de35-4c05-bfd0-d530e71a300a service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:39.606026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:39.676596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:39.677430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:39.677739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:39.678159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] PATH_INFO: `/v3/users/6ca1df33e0204a6cba9501a181561402` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:39.682631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca1df33e0204a6cba9501a181561402)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:39.682631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:39.756927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca1df33e0204a6cba9501a181561402'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.761319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca1df33e0204a6cba9501a181561402'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.781073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca1df33e0204a6cba9501a181561402'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.788904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca1df33e0204a6cba9501a181561402'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.798965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f9980df8-bc4a-4600-873f-581a77b7ff63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca1df33e0204a6cba9501a181561402'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:39.806462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 984/1968] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:39 2022] DELETE /identity/v3/users/6ca1df33e0204a6cba9501a181561402 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:39.987714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:39.989275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:40.018293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f4bcf-aef1-47f2-8783-0793f4a34462 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.080556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.081197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.081473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.081754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] PATH_INFO: `/v3/projects/eacd0759bc654e1b9fffbc0f67f34a5d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.082934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eacd0759bc654e1b9fffbc0f67f34a5d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.084875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:40.136053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eacd0759bc654e1b9fffbc0f67f34a5d', 'request_id': 'req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:40.143628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eacd0759bc654e1b9fffbc0f67f34a5d', 'request_id': 'req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:40.152067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eacd0759bc654e1b9fffbc0f67f34a5d', 'request_id': 'req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:40.154069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:40.165574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] The token cache is being invalidate because project eacd0759bc654e1b9fffbc0f67f34a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:40.166016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:40.166708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-34ace2d4-6cc1-4c9a-b863-67a31d2b586a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:40.168490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 985/1969] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:39 2022] DELETE /identity/v3/projects/eacd0759bc654e1b9fffbc0f67f34a5d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:40.174510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1191ef4-7dbf-444f-a5b9-4e514161a9d9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.174804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1191ef4-7dbf-444f-a5b9-4e514161a9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.175081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1191ef4-7dbf-444f-a5b9-4e514161a9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.195790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f1191ef4-7dbf-444f-a5b9-4e514161a9d9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:40.197023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f1191ef4-7dbf-444f-a5b9-4e514161a9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:40.264022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 985/1970] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:40.271284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abef90-b10f-4acd-8ebc-e9e4fa9fcd33 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.271572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abef90-b10f-4acd-8ebc-e9e4fa9fcd33 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.271840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abef90-b10f-4acd-8ebc-e9e4fa9fcd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.293453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-37abef90-b10f-4acd-8ebc-e9e4fa9fcd33 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:40.294681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-37abef90-b10f-4acd-8ebc-e9e4fa9fcd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:40.303877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 986/1971] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:40.308141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1191ef4-7dbf-444f-a5b9-4e514161a9d9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:40.319138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1191ef4-7dbf-444f-a5b9-4e514161a9d9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.324984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.325618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.325910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.326200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.327114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:40.327467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.329078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:40.342333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 986/1972] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:40.348098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abef90-b10f-4acd-8ebc-e9e4fa9fcd33 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:40.360762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abef90-b10f-4acd-8ebc-e9e4fa9fcd33 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.368575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda0255-9542-46fd-b593-d09291a7bec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.369419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0255-9542-46fd-b593-d09291a7bec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.369791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0255-9542-46fd-b593-d09291a7bec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.370158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0255-9542-46fd-b593-d09291a7bec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.371113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda0255-9542-46fd-b593-d09291a7bec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.373948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda0255-9542-46fd-b593-d09291a7bec9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:40.405559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 987/1973] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:40 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:40.410174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:40.424298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83696b-e85f-4c76-bab6-3237d480bace admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.440925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:40.492569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd2a2-f8ad-4644-b84b-235589bc6f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.493190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd2a2-f8ad-4644-b84b-235589bc6f2c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.493469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd2a2-f8ad-4644-b84b-235589bc6f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.493743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd2a2-f8ad-4644-b84b-235589bc6f2c admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.494378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acd2a2-f8ad-4644-b84b-235589bc6f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.496368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acd2a2-f8ad-4644-b84b-235589bc6f2c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:40.520482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 987/1974] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:40 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:40.524776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda0255-9542-46fd-b593-d09291a7bec9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:40.535426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda0255-9542-46fd-b593-d09291a7bec9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.541142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d58a6-a8b8-4bae-8cfd-8840efbfc12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.541758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d58a6-a8b8-4bae-8cfd-8840efbfc12a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.542034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d58a6-a8b8-4bae-8cfd-8840efbfc12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.542301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d58a6-a8b8-4bae-8cfd-8840efbfc12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.542948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d58a6-a8b8-4bae-8cfd-8840efbfc12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.544579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d58a6-a8b8-4bae-8cfd-8840efbfc12a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:40.558327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 988/1975] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:40.564263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd2a2-f8ad-4644-b84b-235589bc6f2c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:40.575521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd2a2-f8ad-4644-b84b-235589bc6f2c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.581015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68d12f-7be8-4eb8-9d79-67e2848943e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.581686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d12f-7be8-4eb8-9d79-67e2848943e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.581931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d12f-7be8-4eb8-9d79-67e2848943e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.582198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d12f-7be8-4eb8-9d79-67e2848943e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.582828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68d12f-7be8-4eb8-9d79-67e2848943e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.584466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68d12f-7be8-4eb8-9d79-67e2848943e2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:40.595577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 988/1976] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:40.599906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d58a6-a8b8-4bae-8cfd-8840efbfc12a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:40.611671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d58a6-a8b8-4bae-8cfd-8840efbfc12a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.617854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.618564 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.618828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.619079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] PATH_INFO: `/v3/projects/13676f5349c74c88bee06d8de6b007ae/users/e17bfefc0551419a8c8c44a4f15f935d/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.635434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13676f5349c74c88bee06d8de6b007ae, user_id=e17bfefc0551419a8c8c44a4f15f935d, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.638165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:40.652127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 989/1977] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:40 2022] PUT /identity/v3/projects/13676f5349c74c88bee06d8de6b007ae/users/e17bfefc0551419a8c8c44a4f15f935d/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:40.664427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92101ac9-ca65-4baf-8193-ba002ca985c6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.664813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92101ac9-ca65-4baf-8193-ba002ca985c6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.665163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92101ac9-ca65-4baf-8193-ba002ca985c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.702827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-92101ac9-ca65-4baf-8193-ba002ca985c6 None None] MFA Rules not processed for user `e17bfefc0551419a8c8c44a4f15f935d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:40.704095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-92101ac9-ca65-4baf-8193-ba002ca985c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:40.775789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 989/1978] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:40.787032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:40.788817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:40.850376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b92d1-a14a-41ca-8d2a-d4227a3c9618 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.854444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d5c88-65bb-4032-93e0-c5e06c0e66e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.855055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5c88-65bb-4032-93e0-c5e06c0e66e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.855361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5c88-65bb-4032-93e0-c5e06c0e66e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.855640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5c88-65bb-4032-93e0-c5e06c0e66e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.866006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d5c88-65bb-4032-93e0-c5e06c0e66e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.867494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d5c88-65bb-4032-93e0-c5e06c0e66e5 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:40.877529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 990/1979] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:40.898658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92101ac9-ca65-4baf-8193-ba002ca985c6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:40.965494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92101ac9-ca65-4baf-8193-ba002ca985c6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:40.971258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:40.971915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:40.972196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:40.972470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] PATH_INFO: `/v3/users/e17bfefc0551419a8c8c44a4f15f935d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:40.973632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17bfefc0551419a8c8c44a4f15f935d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:40.975605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:41.008307 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:41.038977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17bfefc0551419a8c8c44a4f15f935d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.039698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17bfefc0551419a8c8c44a4f15f935d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.055970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17bfefc0551419a8c8c44a4f15f935d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.063640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17bfefc0551419a8c8c44a4f15f935d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.073262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17bfefc0551419a8c8c44a4f15f935d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.081827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 990/1980] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:40 2022] DELETE /identity/v3/users/e17bfefc0551419a8c8c44a4f15f935d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:41.086320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ddef-4b8e-427d-9d9c-fbaabf0634e6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:41.086645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ddef-4b8e-427d-9d9c-fbaabf0634e6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:41.086893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ddef-4b8e-427d-9d9c-fbaabf0634e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:41.106477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cbf0ddef-4b8e-427d-9d9c-fbaabf0634e6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:41.107819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cbf0ddef-4b8e-427d-9d9c-fbaabf0634e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:41.179782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 991/1981] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:41.190475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:41.192305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:41.260727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6608df-16c5-4d0a-9e65-123e2d4ec0ea admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:41.264914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dca826-b663-47cd-8380-ceb269afe7e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:41.265520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dca826-b663-47cd-8380-ceb269afe7e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:41.265797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dca826-b663-47cd-8380-ceb269afe7e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:41.266064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dca826-b663-47cd-8380-ceb269afe7e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:41.276035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dca826-b663-47cd-8380-ceb269afe7e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:41.277479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dca826-b663-47cd-8380-ceb269afe7e8 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:41.288617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 991/1982] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:41.583719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0ddef-4b8e-427d-9d9c-fbaabf0634e6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:41.585409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cbf0ddef-4b8e-427d-9d9c-fbaabf0634e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:41.603178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0ddef-4b8e-427d-9d9c-fbaabf0634e6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:41.615887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:41.616534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:41.616814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:41.617088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] PATH_INFO: `/v3/projects/13676f5349c74c88bee06d8de6b007ae` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:41.618224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=13676f5349c74c88bee06d8de6b007ae)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:41.620177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:41.655512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:41.679136 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13676f5349c74c88bee06d8de6b007ae', 'request_id': 'req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.688302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13676f5349c74c88bee06d8de6b007ae', 'request_id': 'req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.697016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13676f5349c74c88bee06d8de6b007ae', 'request_id': 'req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.708824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] The token cache is being invalidate because project 13676f5349c74c88bee06d8de6b007ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:41.709114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.709747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b08c8cde-9e97-4a54-91ac-0e5d82cbc3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:41.711279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 992/1983] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:41 2022] DELETE /identity/v3/projects/13676f5349c74c88bee06d8de6b007ae => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:41.716936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2a515-d832-47e4-9203-b9cb7b7ed7b8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:41.717671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2a515-d832-47e4-9203-b9cb7b7ed7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:41.717671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2a515-d832-47e4-9203-b9cb7b7ed7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:41.741414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4fc2a515-d832-47e4-9203-b9cb7b7ed7b8 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:41.742643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fc2a515-d832-47e4-9203-b9cb7b7ed7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:41.805156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 992/1984] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:41.813305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d268e-ce14-4010-8a28-df265804b2f1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:41.813587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d268e-ce14-4010-8a28-df265804b2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:41.813885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d268e-ce14-4010-8a28-df265804b2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:41.835531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9d1d268e-ce14-4010-8a28-df265804b2f1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:41.836961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9d1d268e-ce14-4010-8a28-df265804b2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:41.845889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 993/1985] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:41.850265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2a515-d832-47e4-9203-b9cb7b7ed7b8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:41.861088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2a515-d832-47e4-9203-b9cb7b7ed7b8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:41.866024 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:41.866648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:41.866995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:41.867340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:41.868151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:41.868323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:41.869956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:41.885998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 993/1986] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:19:41.887679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d268e-ce14-4010-8a28-df265804b2f1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:41.898034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d268e-ce14-4010-8a28-df265804b2f1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:41.904680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e509e-a6f5-4dee-877e-be0f81d6d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:41.905339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e509e-a6f5-4dee-877e-be0f81d6d6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:41.905738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e509e-a6f5-4dee-877e-be0f81d6d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:41.906071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e509e-a6f5-4dee-877e-be0f81d6d6b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:41.907018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e509e-a6f5-4dee-877e-be0f81d6d6b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:41.909538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e509e-a6f5-4dee-877e-be0f81d6d6b4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:41.963912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 994/1987] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:41 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:41.970303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:41.988936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f73bc-c5aa-472d-bd54-94730a42d6b6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:42.029866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:42.077752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69004dfa-7662-454e-892c-bf204a5ba851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:42.078480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004dfa-7662-454e-892c-bf204a5ba851 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:42.078764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004dfa-7662-454e-892c-bf204a5ba851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:42.079142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004dfa-7662-454e-892c-bf204a5ba851 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:42.080394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69004dfa-7662-454e-892c-bf204a5ba851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:42.083487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69004dfa-7662-454e-892c-bf204a5ba851 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:42.111610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 994/1988] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:41 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:42.112777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e509e-a6f5-4dee-877e-be0f81d6d6b4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:42.123746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e509e-a6f5-4dee-877e-be0f81d6d6b4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:42.128864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ca09-6570-4cfd-a4e3-8792aadf592a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:42.129483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ca09-6570-4cfd-a4e3-8792aadf592a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:42.130289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ca09-6570-4cfd-a4e3-8792aadf592a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:42.130289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ca09-6570-4cfd-a4e3-8792aadf592a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:42.130845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ca09-6570-4cfd-a4e3-8792aadf592a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:42.133005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ca09-6570-4cfd-a4e3-8792aadf592a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:42.150091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 995/1989] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:42.151404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69004dfa-7662-454e-892c-bf204a5ba851 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:42.169582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69004dfa-7662-454e-892c-bf204a5ba851 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:42.176010 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a7bea-83ef-4dde-882e-217c0e156d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:42.176863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7bea-83ef-4dde-882e-217c0e156d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:42.176863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7bea-83ef-4dde-882e-217c0e156d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:42.177148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7bea-83ef-4dde-882e-217c0e156d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:42.177831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a7bea-83ef-4dde-882e-217c0e156d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:42.179499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a7bea-83ef-4dde-882e-217c0e156d1e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:42.202235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 995/1990] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:42.202565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ca09-6570-4cfd-a4e3-8792aadf592a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:42.217130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ca09-6570-4cfd-a4e3-8792aadf592a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:42.225461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:42.226696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:42.226696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:42.226970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] PATH_INFO: `/v3/projects/6b0a57934c5b43e1b92aafb68796efcb/users/c2a8336f29434204ae878d92d86196bf/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:42.255169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0a57934c5b43e1b92aafb68796efcb, user_id=c2a8336f29434204ae878d92d86196bf, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:42.258172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:42.287331 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 996/1991] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:42 2022] PUT /identity/v3/projects/6b0a57934c5b43e1b92aafb68796efcb/users/c2a8336f29434204ae878d92d86196bf/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:42.293523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e98409-bbc0-4b0b-b1fe-90634adef0aa None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:42.293523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e98409-bbc0-4b0b-b1fe-90634adef0aa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:42.293826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e98409-bbc0-4b0b-b1fe-90634adef0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:42.334254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d9e98409-bbc0-4b0b-b1fe-90634adef0aa None None] MFA Rules not processed for user `c2a8336f29434204ae878d92d86196bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:42.335267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9e98409-bbc0-4b0b-b1fe-90634adef0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:42.425247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 996/1992] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:42.438551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:42.440584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:42.567203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584ba25-86ed-4bed-8068-3908cf482832 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:42.574119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57e2d7-f212-457e-ba86-b81d6a055ac2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:42.575103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57e2d7-f212-457e-ba86-b81d6a055ac2 service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:42.575607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57e2d7-f212-457e-ba86-b81d6a055ac2 service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:42.576130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57e2d7-f212-457e-ba86-b81d6a055ac2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:42.589528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57e2d7-f212-457e-ba86-b81d6a055ac2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:42.593142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57e2d7-f212-457e-ba86-b81d6a055ac2 service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:42.608806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 997/1993] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:44.195553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e98409-bbc0-4b0b-b1fe-90634adef0aa None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:44.196960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9e98409-bbc0-4b0b-b1fe-90634adef0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:44.278514 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e98409-bbc0-4b0b-b1fe-90634adef0aa None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:44.283174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:44.283903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:44.284162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:44.284429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] PATH_INFO: `/v3/users/942c167ca1ba466cbf2ded0f4ab6b2e2` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:44.285531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=942c167ca1ba466cbf2ded0f4ab6b2e2)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:44.287561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:44.346794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:44.349652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942c167ca1ba466cbf2ded0f4ab6b2e2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:44.350292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942c167ca1ba466cbf2ded0f4ab6b2e2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:44.366402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942c167ca1ba466cbf2ded0f4ab6b2e2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:44.373831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942c167ca1ba466cbf2ded0f4ab6b2e2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:44.382711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942c167ca1ba466cbf2ded0f4ab6b2e2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:44.388830 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 997/1994] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:44 2022] DELETE /identity/v3/users/942c167ca1ba466cbf2ded0f4ab6b2e2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:44.393497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f0020-6956-4c06-a5f3-770a637b9125 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:44.394050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f0020-6956-4c06-a5f3-770a637b9125 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:44.394145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f0020-6956-4c06-a5f3-770a637b9125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:44.418822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1c2f0020-6956-4c06-a5f3-770a637b9125 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:44.420339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1c2f0020-6956-4c06-a5f3-770a637b9125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:44.469878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:44.485761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 998/1995] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:44.499172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:44.501082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:44.569485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4317b-4545-4bd2-827f-06f7fec7c810 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:44.573413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554efa08-8a1a-427d-a2b5-e5e0b419e50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:44.574019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554efa08-8a1a-427d-a2b5-e5e0b419e50a service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:44.574292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554efa08-8a1a-427d-a2b5-e5e0b419e50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:44.574590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554efa08-8a1a-427d-a2b5-e5e0b419e50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:44.585385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554efa08-8a1a-427d-a2b5-e5e0b419e50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:44.586985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554efa08-8a1a-427d-a2b5-e5e0b419e50a service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:44.600878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 998/1996] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:45.064369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f0020-6956-4c06-a5f3-770a637b9125 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.066319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1c2f0020-6956-4c06-a5f3-770a637b9125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:45.095386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f0020-6956-4c06-a5f3-770a637b9125 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:45.100900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:45.101759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.101983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.102419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] PATH_INFO: `/v3/projects/b804356a5b6a45eb9789286a23b69e0d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.104625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b804356a5b6a45eb9789286a23b69e0d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:45.107481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:45.167833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b804356a5b6a45eb9789286a23b69e0d', 'request_id': 'req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.175123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b804356a5b6a45eb9789286a23b69e0d', 'request_id': 'req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.184655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b804356a5b6a45eb9789286a23b69e0d', 'request_id': 'req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.198786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] The token cache is being invalidate because project b804356a5b6a45eb9789286a23b69e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:45.199063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.199753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d6c5b2b5-9f13-4ebc-aa29-affe166b4c0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.201426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 999/1997] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:45 2022] DELETE /identity/v3/projects/b804356a5b6a45eb9789286a23b69e0d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:45.208139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43aabc-e132-431e-adac-135081448848 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.208712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43aabc-e132-431e-adac-135081448848 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.208712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43aabc-e132-431e-adac-135081448848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.230762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7f43aabc-e132-431e-adac-135081448848 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:45.232222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7f43aabc-e132-431e-adac-135081448848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:45.320385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 999/1998] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:45.322244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fffec8-1bcc-4285-af0c-34292f2382b2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.322668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fffec8-1bcc-4285-af0c-34292f2382b2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.322935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fffec8-1bcc-4285-af0c-34292f2382b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.351350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d3fffec8-1bcc-4285-af0c-34292f2382b2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:45.353115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d3fffec8-1bcc-4285-af0c-34292f2382b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:45.364686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1000/1999] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:45.368605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43aabc-e132-431e-adac-135081448848 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.382208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43aabc-e132-431e-adac-135081448848 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:45.389271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:45.390190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.391223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.391825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.393063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:45.393540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:45.396436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:45.414876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1000/2000] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:45.421307 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fffec8-1bcc-4285-af0c-34292f2382b2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.433193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fffec8-1bcc-4285-af0c-34292f2382b2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:45.440549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23ef03-76c5-4c22-a813-993e3503f71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:45.441712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ef03-76c5-4c22-a813-993e3503f71b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.441712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ef03-76c5-4c22-a813-993e3503f71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.441924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ef03-76c5-4c22-a813-993e3503f71b admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.442454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23ef03-76c5-4c22-a813-993e3503f71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:45.444413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23ef03-76c5-4c22-a813-993e3503f71b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:45.481053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1001/2001] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:45 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:45.484753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.496403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0e8c0-9d03-4822-948c-66ae23f40e90 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:45.565564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:45.581807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c266c5-c801-4768-8d7a-cda57a8cd904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:45.582915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c266c5-c801-4768-8d7a-cda57a8cd904 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.583518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c266c5-c801-4768-8d7a-cda57a8cd904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.583903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c266c5-c801-4768-8d7a-cda57a8cd904 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.584752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c266c5-c801-4768-8d7a-cda57a8cd904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:45.587386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c266c5-c801-4768-8d7a-cda57a8cd904 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:45.615548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1001/2002] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:45 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:45.619796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23ef03-76c5-4c22-a813-993e3503f71b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.636941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23ef03-76c5-4c22-a813-993e3503f71b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:45.646527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeab8fb-0ee0-4c8b-b0b2-818c78244cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:45.647695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeab8fb-0ee0-4c8b-b0b2-818c78244cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.648176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeab8fb-0ee0-4c8b-b0b2-818c78244cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.649621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeab8fb-0ee0-4c8b-b0b2-818c78244cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.650599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeab8fb-0ee0-4c8b-b0b2-818c78244cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:45.652563 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeab8fb-0ee0-4c8b-b0b2-818c78244cd1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:45.672504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1002/2003] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:45.679892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c266c5-c801-4768-8d7a-cda57a8cd904 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.696863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c266c5-c801-4768-8d7a-cda57a8cd904 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:45.709815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:45.711418 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.711785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.712246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.712939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:45.714877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:45.731994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1002/2004] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:45.739711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeab8fb-0ee0-4c8b-b0b2-818c78244cd1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.745670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.747805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:45.756540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeab8fb-0ee0-4c8b-b0b2-818c78244cd1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:45.761730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8427ff1-dde8-48cc-899c-513c2d7b0d8f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:45.767519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:45.768265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.768596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28509a42-2170-4f3f-bffa-43593f2fa4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:45.768748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.768999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] PATH_INFO: `/v3/users/dc79a224290b4d9ba30aebe72086c21d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.769478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28509a42-2170-4f3f-bffa-43593f2fa4da admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.769858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28509a42-2170-4f3f-bffa-43593f2fa4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.770168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28509a42-2170-4f3f-bffa-43593f2fa4da admin admin] PATH_INFO: `/v3/projects/2188dd8748d540ee8df3689bd8538bc8/users/0987854672d0472ea9dfdfa2d1900a3b/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.770515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc79a224290b4d9ba30aebe72086c21d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:45.772969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:45.790031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28509a42-2170-4f3f-bffa-43593f2fa4da admin admin] RBAC: Authorizing `identity:create_grant(project_id=2188dd8748d540ee8df3689bd8538bc8, user_id=0987854672d0472ea9dfdfa2d1900a3b, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:45.793057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28509a42-2170-4f3f-bffa-43593f2fa4da admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:45.810832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1003/2005] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:45 2022] PUT /identity/v3/projects/2188dd8748d540ee8df3689bd8538bc8/users/0987854672d0472ea9dfdfa2d1900a3b/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:45.815869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b579f-c65e-46b6-a507-2589741d2c5b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.816396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b579f-c65e-46b6-a507-2589741d2c5b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.816970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b579f-c65e-46b6-a507-2589741d2c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.838796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc79a224290b4d9ba30aebe72086c21d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.839316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f70b579f-c65e-46b6-a507-2589741d2c5b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:45.839726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc79a224290b4d9ba30aebe72086c21d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.840544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f70b579f-c65e-46b6-a507-2589741d2c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:45.854301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc79a224290b4d9ba30aebe72086c21d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.867263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc79a224290b4d9ba30aebe72086c21d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.882553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af9424a-bbef-4573-af26-af346232af1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc79a224290b4d9ba30aebe72086c21d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:45.900369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1003/2006] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:45 2022] DELETE /identity/v3/users/dc79a224290b4d9ba30aebe72086c21d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:19:45.903183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa364a4-5305-423e-94e5-6274d6436534 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:45.903183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa364a4-5305-423e-94e5-6274d6436534 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:45.903183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa364a4-5305-423e-94e5-6274d6436534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:45.926237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1004/2007] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:45.931936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4fa364a4-5305-423e-94e5-6274d6436534 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:45.933175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fa364a4-5305-423e-94e5-6274d6436534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:45.940599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b579f-c65e-46b6-a507-2589741d2c5b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:45.942362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f70b579f-c65e-46b6-a507-2589741d2c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:45.957825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:46.014170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b579f-c65e-46b6-a507-2589741d2c5b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:46.018083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1004/2008] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:46.028104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa364a4-5305-423e-94e5-6274d6436534 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:46.029851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fa364a4-5305-423e-94e5-6274d6436534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:46.093367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:46.094961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:46.095481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:46.095938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:46.100080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:46.114630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa364a4-5305-423e-94e5-6274d6436534 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:46.119385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:46.121649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:46.125273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beb350-8370-4982-8212-a953dbe1bbdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:46.125919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb350-8370-4982-8212-a953dbe1bbdf service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:46.126199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb350-8370-4982-8212-a953dbe1bbdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:46.126472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb350-8370-4982-8212-a953dbe1bbdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:46.135976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1005/2009] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:46.141824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beb350-8370-4982-8212-a953dbe1bbdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:46.143370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beb350-8370-4982-8212-a953dbe1bbdf service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:46.155331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1005/2010] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:19:46.588190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:46.589905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:46.614622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da209ad2-6188-4da6-9738-77ea1c2bb583 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:46.628236 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:46.629128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:46.629562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:46.629972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] PATH_INFO: `/v3/projects/86e3f044eac8427bbba518b6b2ef45eb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:46.631616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e3f044eac8427bbba518b6b2ef45eb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:46.635847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:46.695180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e3f044eac8427bbba518b6b2ef45eb', 'request_id': 'req-4259cad7-478b-4b9a-8429-3d22490e9910'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:46.703834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e3f044eac8427bbba518b6b2ef45eb', 'request_id': 'req-4259cad7-478b-4b9a-8429-3d22490e9910'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:46.718640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e3f044eac8427bbba518b6b2ef45eb', 'request_id': 'req-4259cad7-478b-4b9a-8429-3d22490e9910'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:46.731823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] The token cache is being invalidate because project 86e3f044eac8427bbba518b6b2ef45eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:46.732324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:46.733280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:46.734958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1006/2011] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:46 2022] DELETE /identity/v3/projects/86e3f044eac8427bbba518b6b2ef45eb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:46.740155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beb350-8370-4982-8212-a953dbe1bbdf service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:46.752099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beb350-8370-4982-8212-a953dbe1bbdf service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:46.830079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:46.831765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:46.831765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:46.831765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] PATH_INFO: `/v3/users/f2f6ecf739ac420eadaf1ddfb899fa10` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:46.832611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f6ecf739ac420eadaf1ddfb899fa10)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:46.834589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:46.950002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f6ecf739ac420eadaf1ddfb899fa10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:46.950806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f6ecf739ac420eadaf1ddfb899fa10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:46.968885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f6ecf739ac420eadaf1ddfb899fa10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:46.981385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f6ecf739ac420eadaf1ddfb899fa10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:47.001404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6a38247d-08d0-4626-8d33-bcd1b411b209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f6ecf739ac420eadaf1ddfb899fa10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:47.024557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1006/2012] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:46 2022] DELETE /identity/v3/users/f2f6ecf739ac420eadaf1ddfb899fa10 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:47.433358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:47.434804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:47.464395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259cad7-478b-4b9a-8429-3d22490e9910 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:47.546847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:47.583352 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:47.584483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:47.585073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:47.585527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] PATH_INFO: `/v3/projects/303e900f1c1945deb2c3f472188f1e35` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:47.587010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=303e900f1c1945deb2c3f472188f1e35)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:47.589337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:47.652305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303e900f1c1945deb2c3f472188f1e35', 'request_id': 'req-57deefe6-fe35-40e6-8857-4349c917f364'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:47.659757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303e900f1c1945deb2c3f472188f1e35', 'request_id': 'req-57deefe6-fe35-40e6-8857-4349c917f364'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:47.672178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303e900f1c1945deb2c3f472188f1e35', 'request_id': 'req-57deefe6-fe35-40e6-8857-4349c917f364'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:47.689908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] The token cache is being invalidate because project 303e900f1c1945deb2c3f472188f1e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:47.690549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:47.691637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-57deefe6-fe35-40e6-8857-4349c917f364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:47.693810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1007/2013] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:47 2022] DELETE /identity/v3/projects/303e900f1c1945deb2c3f472188f1e35 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:47.700920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932bbcd-1eaf-4e12-b33e-632414726aa2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:47.701328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932bbcd-1eaf-4e12-b33e-632414726aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:47.701671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932bbcd-1eaf-4e12-b33e-632414726aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:47.731767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1932bbcd-1eaf-4e12-b33e-632414726aa2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:47.733406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1932bbcd-1eaf-4e12-b33e-632414726aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:47.835438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1007/2014] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:47.836907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d138bc-4ae7-4378-924a-79c626985e45 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:47.837355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d138bc-4ae7-4378-924a-79c626985e45 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:47.837502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d138bc-4ae7-4378-924a-79c626985e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:47.907474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c2d138bc-4ae7-4378-924a-79c626985e45 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:47.908842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c2d138bc-4ae7-4378-924a-79c626985e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:47.931981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1008/2015] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:47.936318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932bbcd-1eaf-4e12-b33e-632414726aa2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:47.949684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932bbcd-1eaf-4e12-b33e-632414726aa2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:47.957732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:47.958375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:47.958739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:47.958988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:47.959713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:47.960021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:47.967517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:47.971676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:47.999580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1008/2016] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:19:48.004208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d138bc-4ae7-4378-924a-79c626985e45 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:48.017039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d138bc-4ae7-4378-924a-79c626985e45 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:48.023468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a6c5d-2f32-4cbf-9206-d411489b9c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:48.024340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a6c5d-2f32-4cbf-9206-d411489b9c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.024623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a6c5d-2f32-4cbf-9206-d411489b9c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.024896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a6c5d-2f32-4cbf-9206-d411489b9c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.025545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a6c5d-2f32-4cbf-9206-d411489b9c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:48.029743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a6c5d-2f32-4cbf-9206-d411489b9c8a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:48.101077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1009/2017] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:48 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:48.102287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:48.119649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60540c68-c146-4bbf-890c-b01e9a6b0022 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:48.225861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e22aa-7698-4efb-8302-6b093d11baed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:48.226616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e22aa-7698-4efb-8302-6b093d11baed admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.226757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e22aa-7698-4efb-8302-6b093d11baed admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.227030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e22aa-7698-4efb-8302-6b093d11baed admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.227792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e22aa-7698-4efb-8302-6b093d11baed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:48.229765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e22aa-7698-4efb-8302-6b093d11baed admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:48.266690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1009/2018] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:48 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:48.270974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a6c5d-2f32-4cbf-9206-d411489b9c8a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:48.285193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a6c5d-2f32-4cbf-9206-d411489b9c8a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:48.296423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007acb5-c826-41e5-9ff1-511f05f5ef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:48.297502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007acb5-c826-41e5-9ff1-511f05f5ef3e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.297970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007acb5-c826-41e5-9ff1-511f05f5ef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.298723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007acb5-c826-41e5-9ff1-511f05f5ef3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.300068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007acb5-c826-41e5-9ff1-511f05f5ef3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:48.302280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007acb5-c826-41e5-9ff1-511f05f5ef3e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:48.322042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1010/2019] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:48.325599 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e22aa-7698-4efb-8302-6b093d11baed admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:48.341995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e22aa-7698-4efb-8302-6b093d11baed admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:48.347924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca263-236a-44f1-a9ca-5a8182d84e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:48.355682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca263-236a-44f1-a9ca-5a8182d84e92 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.356451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca263-236a-44f1-a9ca-5a8182d84e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.357070 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca263-236a-44f1-a9ca-5a8182d84e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.359965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca263-236a-44f1-a9ca-5a8182d84e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:48.363378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca263-236a-44f1-a9ca-5a8182d84e92 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:48.383184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1010/2020] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:48.384515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007acb5-c826-41e5-9ff1-511f05f5ef3e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:48.397367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007acb5-c826-41e5-9ff1-511f05f5ef3e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:48.402698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af98b0-d11d-496d-aa09-018818cf9cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:48.403447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af98b0-d11d-496d-aa09-018818cf9cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.403723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af98b0-d11d-496d-aa09-018818cf9cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.404255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af98b0-d11d-496d-aa09-018818cf9cf8 admin admin] PATH_INFO: `/v3/projects/d6318ae5624a4874abeb00eff1879a62/users/b4fa3971998548dc8c82d821a02ef3d4/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.427425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af98b0-d11d-496d-aa09-018818cf9cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6318ae5624a4874abeb00eff1879a62, user_id=b4fa3971998548dc8c82d821a02ef3d4, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:48.432252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af98b0-d11d-496d-aa09-018818cf9cf8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:48.449520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1011/2021] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:48 2022] PUT /identity/v3/projects/d6318ae5624a4874abeb00eff1879a62/users/b4fa3971998548dc8c82d821a02ef3d4/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:48.461429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e6f28-051d-4f02-8e4f-715a41195759 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.461929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e6f28-051d-4f02-8e4f-715a41195759 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.462365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e6f28-051d-4f02-8e4f-715a41195759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.514593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-950e6f28-051d-4f02-8e4f-715a41195759 None None] MFA Rules not processed for user `b4fa3971998548dc8c82d821a02ef3d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:48.516414 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-950e6f28-051d-4f02-8e4f-715a41195759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:48.613903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1011/2022] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:48.618754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af98b0-d11d-496d-aa09-018818cf9cf8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:48.722968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af98b0-d11d-496d-aa09-018818cf9cf8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:48.728973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f9c5d-452d-43a7-8c8a-c60529ee9ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:48.729876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f9c5d-452d-43a7-8c8a-c60529ee9ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.730442 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f9c5d-452d-43a7-8c8a-c60529ee9ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.730869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f9c5d-452d-43a7-8c8a-c60529ee9ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.731794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f9c5d-452d-43a7-8c8a-c60529ee9ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:48.734176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f9c5d-452d-43a7-8c8a-c60529ee9ddc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:48.783012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1012/2023] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:48 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:48.786777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e6f28-051d-4f02-8e4f-715a41195759 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:48.802055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e6f28-051d-4f02-8e4f-715a41195759 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:48.905337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f8170-b993-43e1-877b-d3c46228843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:48.906215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f8170-b993-43e1-877b-d3c46228843b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.906660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f8170-b993-43e1-877b-d3c46228843b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.908591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f8170-b993-43e1-877b-d3c46228843b admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.908591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f8170-b993-43e1-877b-d3c46228843b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:48.912012 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f8170-b993-43e1-877b-d3c46228843b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:48.929171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:48.963843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1012/2024] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:48 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:48.969306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f9c5d-452d-43a7-8c8a-c60529ee9ddc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:48.982678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f9c5d-452d-43a7-8c8a-c60529ee9ddc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:48.991692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fa13c-fdbf-43bd-9608-14f4e2758ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:48.992434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fa13c-fdbf-43bd-9608-14f4e2758ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:48.992809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fa13c-fdbf-43bd-9608-14f4e2758ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:48.993103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fa13c-fdbf-43bd-9608-14f4e2758ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:48.993795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156fa13c-fdbf-43bd-9608-14f4e2758ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:48.995688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156fa13c-fdbf-43bd-9608-14f4e2758ae7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:49.010854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:49.027333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1013/2025] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:19:49.028751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f8170-b993-43e1-877b-d3c46228843b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:49.060005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f8170-b993-43e1-877b-d3c46228843b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:49.074577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b4fd-83ef-456b-9bf0-7d627fa7c983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:49.076846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b4fd-83ef-456b-9bf0-7d627fa7c983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:49.077437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b4fd-83ef-456b-9bf0-7d627fa7c983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:49.077982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b4fd-83ef-456b-9bf0-7d627fa7c983 admin admin] PATH_INFO: `/v3/projects/221c1a5fd69b43b194a4cf34994ab99f/users/0848b173e74f4fef91a1e9d065178af4/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:49.100837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b4fd-83ef-456b-9bf0-7d627fa7c983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221c1a5fd69b43b194a4cf34994ab99f, user_id=0848b173e74f4fef91a1e9d065178af4, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:49.104401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b4fd-83ef-456b-9bf0-7d627fa7c983 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:49.131515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1013/2026] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:49 2022] PUT /identity/v3/projects/221c1a5fd69b43b194a4cf34994ab99f/users/0848b173e74f4fef91a1e9d065178af4/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:49.140180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fa13c-fdbf-43bd-9608-14f4e2758ae7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:49.174903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fa13c-fdbf-43bd-9608-14f4e2758ae7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:49.289950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729340a6-fb86-4b87-ac26-27a19cc171ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:49.290619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729340a6-fb86-4b87-ac26-27a19cc171ce admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:49.290926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729340a6-fb86-4b87-ac26-27a19cc171ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:49.291344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729340a6-fb86-4b87-ac26-27a19cc171ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:49.292017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729340a6-fb86-4b87-ac26-27a19cc171ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:49.295770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729340a6-fb86-4b87-ac26-27a19cc171ce admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:49.322930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1014/2027] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:49.325408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b4fd-83ef-456b-9bf0-7d627fa7c983 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:49.344751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b4fd-83ef-456b-9bf0-7d627fa7c983 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:49.354025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:49.355147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:49.355319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:49.355546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] PATH_INFO: `/v3/projects/221c1a5fd69b43b194a4cf34994ab99f/users/0848b173e74f4fef91a1e9d065178af4/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:49.360251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221c1a5fd69b43b194a4cf34994ab99f, user_id=0848b173e74f4fef91a1e9d065178af4, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:49.362961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:49.377941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1014/2028] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:49 2022] PUT /identity/v3/projects/221c1a5fd69b43b194a4cf34994ab99f/users/0848b173e74f4fef91a1e9d065178af4/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:49.392375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb6bb8-fa8c-4139-9da0-4280e8a1184b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:49.393355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb6bb8-fa8c-4139-9da0-4280e8a1184b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:49.393355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb6bb8-fa8c-4139-9da0-4280e8a1184b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:49.429940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1ddb6bb8-fa8c-4139-9da0-4280e8a1184b None None] MFA Rules not processed for user `0848b173e74f4fef91a1e9d065178af4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:49.432025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1ddb6bb8-fa8c-4139-9da0-4280e8a1184b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:49.514466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1015/2029] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:49.527631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:49.532487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:49.615355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d27e2-05a7-4775-a202-5f7d661cd055 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:49.622790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b6cf-82ff-4024-b770-700537a97966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:49.623825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b6cf-82ff-4024-b770-700537a97966 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:49.624431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b6cf-82ff-4024-b770-700537a97966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:49.624797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b6cf-82ff-4024-b770-700537a97966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:49.637739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600b6cf-82ff-4024-b770-700537a97966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:49.640249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600b6cf-82ff-4024-b770-700537a97966 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:49.652708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1015/2030] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:51.990433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb6bb8-fa8c-4139-9da0-4280e8a1184b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:52.133035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb6bb8-fa8c-4139-9da0-4280e8a1184b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:52.140641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:52.141792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:52.142462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:52.142956 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] PATH_INFO: `/v3/users/b4fa3971998548dc8c82d821a02ef3d4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:52.144429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fa3971998548dc8c82d821a02ef3d4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:52.152280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:52.212681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:52.317898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fa3971998548dc8c82d821a02ef3d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:52.318270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fa3971998548dc8c82d821a02ef3d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:52.341618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fa3971998548dc8c82d821a02ef3d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:52.342335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fa3971998548dc8c82d821a02ef3d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:52.392791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fa3971998548dc8c82d821a02ef3d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:52.426711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1016/2031] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:51 2022] DELETE /identity/v3/users/b4fa3971998548dc8c82d821a02ef3d4 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:52.429023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04ad65-9d54-4d58-8416-688da49997a9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:52.429483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04ad65-9d54-4d58-8416-688da49997a9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:52.429695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04ad65-9d54-4d58-8416-688da49997a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:52.477344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cb04ad65-9d54-4d58-8416-688da49997a9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:52.478754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cb04ad65-9d54-4d58-8416-688da49997a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:52.561101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1016/2032] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:52.574927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:52.578347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:52.605118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10c9a-c3f2-4474-aa4a-9d8b92f75cb3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:52.605472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10c9a-c3f2-4474-aa4a-9d8b92f75cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:52.605812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10c9a-c3f2-4474-aa4a-9d8b92f75cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:52.669056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-14e10c9a-c3f2-4474-aa4a-9d8b92f75cb3 None None] MFA Rules not processed for user `0987854672d0472ea9dfdfa2d1900a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:52.670371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14e10c9a-c3f2-4474-aa4a-9d8b92f75cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:52.724912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:52.726310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9f3d-6e3c-4a31-8c9e-e22c9070e035 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:52.739333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b0f31-ab86-43d4-8303-3bdbbfb9b887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:52.740342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b0f31-ab86-43d4-8303-3bdbbfb9b887 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:52.740913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b0f31-ab86-43d4-8303-3bdbbfb9b887 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:52.741279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b0f31-ab86-43d4-8303-3bdbbfb9b887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:52.753713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b0f31-ab86-43d4-8303-3bdbbfb9b887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:52.755129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1017/2033] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:52.759016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b0f31-ab86-43d4-8303-3bdbbfb9b887 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:52.759166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e10c9a-c3f2-4474-aa4a-9d8b92f75cb3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:52.762297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14e10c9a-c3f2-4474-aa4a-9d8b92f75cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:52.777960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e10c9a-c3f2-4474-aa4a-9d8b92f75cb3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:52.783201 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1017/2034] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:52.786653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a384f75-20cb-4072-aa1f-35ffa8601172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:52.787306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a384f75-20cb-4072-aa1f-35ffa8601172 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:52.787585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a384f75-20cb-4072-aa1f-35ffa8601172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:52.787855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a384f75-20cb-4072-aa1f-35ffa8601172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:52.788528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a384f75-20cb-4072-aa1f-35ffa8601172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:52.790403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a384f75-20cb-4072-aa1f-35ffa8601172 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:52.840876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1018/2035] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:52 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:52.845420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b0f31-ab86-43d4-8303-3bdbbfb9b887 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:52.860851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b0f31-ab86-43d4-8303-3bdbbfb9b887 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:52.958256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d7e4-f65e-487c-8a1e-f5fad0ac7925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:52.958891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d7e4-f65e-487c-8a1e-f5fad0ac7925 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:52.959166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d7e4-f65e-487c-8a1e-f5fad0ac7925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:52.959484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d7e4-f65e-487c-8a1e-f5fad0ac7925 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:52.960120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d7e4-f65e-487c-8a1e-f5fad0ac7925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:52.962069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d7e4-f65e-487c-8a1e-f5fad0ac7925 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:52.988533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1018/2036] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:52 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:52.992341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a384f75-20cb-4072-aa1f-35ffa8601172 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:53.011643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a384f75-20cb-4072-aa1f-35ffa8601172 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:53.018025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48249cdc-bd67-4b98-b8cf-5d897caa3eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:53.018728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48249cdc-bd67-4b98-b8cf-5d897caa3eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:53.019006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48249cdc-bd67-4b98-b8cf-5d897caa3eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:53.019305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48249cdc-bd67-4b98-b8cf-5d897caa3eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:53.020186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48249cdc-bd67-4b98-b8cf-5d897caa3eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:53.021829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48249cdc-bd67-4b98-b8cf-5d897caa3eaf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:53.035285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1019/2037] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:53.037553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d7e4-f65e-487c-8a1e-f5fad0ac7925 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:53.058047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d7e4-f65e-487c-8a1e-f5fad0ac7925 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:53.065514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02480416-4d6d-413b-af30-4a94c40a8aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:53.066213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02480416-4d6d-413b-af30-4a94c40a8aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:53.066497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02480416-4d6d-413b-af30-4a94c40a8aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:53.066769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02480416-4d6d-413b-af30-4a94c40a8aeb admin admin] PATH_INFO: `/v3/projects/bacc8bf88e20450dbabf1c0f9b337691/users/5b3928d08cfd420280c1eb7bd1e97832/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:53.092433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02480416-4d6d-413b-af30-4a94c40a8aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacc8bf88e20450dbabf1c0f9b337691, user_id=5b3928d08cfd420280c1eb7bd1e97832, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:53.095442 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02480416-4d6d-413b-af30-4a94c40a8aeb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:53.113990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1019/2038] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:53 2022] PUT /identity/v3/projects/bacc8bf88e20450dbabf1c0f9b337691/users/5b3928d08cfd420280c1eb7bd1e97832/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:53.118532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48249cdc-bd67-4b98-b8cf-5d897caa3eaf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:53.130471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48249cdc-bd67-4b98-b8cf-5d897caa3eaf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:53.196540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:53.197165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:53.197443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:53.197711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:53.198357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:53.200076 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:53.211636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1020/2039] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:53.215955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02480416-4d6d-413b-af30-4a94c40a8aeb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:53.223742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:53.229562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02480416-4d6d-413b-af30-4a94c40a8aeb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:53.236925 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:53.237897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:53.238228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:53.238729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] PATH_INFO: `/v3/projects/bacc8bf88e20450dbabf1c0f9b337691/users/5b3928d08cfd420280c1eb7bd1e97832/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:53.241170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacc8bf88e20450dbabf1c0f9b337691, user_id=5b3928d08cfd420280c1eb7bd1e97832, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:53.243965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:53.256715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1020/2040] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:53 2022] PUT /identity/v3/projects/bacc8bf88e20450dbabf1c0f9b337691/users/5b3928d08cfd420280c1eb7bd1e97832/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:53.384535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:53.387046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:53.405994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431d11e-29ae-4abd-9ced-51d8c560dd55 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:53.473297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:53.474309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:53.475103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:53.475103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] PATH_INFO: `/v3/projects/d6318ae5624a4874abeb00eff1879a62` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:53.476411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6318ae5624a4874abeb00eff1879a62)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:53.478490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:53.536241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6318ae5624a4874abeb00eff1879a62', 'request_id': 'req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.538511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:53.548254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6318ae5624a4874abeb00eff1879a62', 'request_id': 'req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.557407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6318ae5624a4874abeb00eff1879a62', 'request_id': 'req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.571223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] The token cache is being invalidate because project d6318ae5624a4874abeb00eff1879a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:53.571558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.572194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.573781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1021/2041] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:53 2022] DELETE /identity/v3/projects/d6318ae5624a4874abeb00eff1879a62 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:53.578301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:53.580386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:53.595670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6858b8-23aa-4cce-bbe4-3e2938edaf10 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:53.684670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:53.686231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:53.687372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:53.688059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] PATH_INFO: `/v3/users/0848b173e74f4fef91a1e9d065178af4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:53.696343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0848b173e74f4fef91a1e9d065178af4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:53.706933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:53.818585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0848b173e74f4fef91a1e9d065178af4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.819532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0848b173e74f4fef91a1e9d065178af4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.830485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0848b173e74f4fef91a1e9d065178af4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.831463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0848b173e74f4fef91a1e9d065178af4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.865054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0cc190b5-1d29-4c22-8d1c-b5196759bc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0848b173e74f4fef91a1e9d065178af4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:53.905368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1021/2042] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:19:53 2022] DELETE /identity/v3/users/0848b173e74f4fef91a1e9d065178af4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:54.371659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:54.388608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069ef07-93b4-44fb-bdd0-086dc2b7dce6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:54.478681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:54.479416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:54.479648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:54.479914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] PATH_INFO: `/v3/projects/221c1a5fd69b43b194a4cf34994ab99f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:54.481131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=221c1a5fd69b43b194a4cf34994ab99f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:54.483780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:54.569679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221c1a5fd69b43b194a4cf34994ab99f', 'request_id': 'req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:54.581434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221c1a5fd69b43b194a4cf34994ab99f', 'request_id': 'req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:54.590338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221c1a5fd69b43b194a4cf34994ab99f', 'request_id': 'req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:54.609744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] The token cache is being invalidate because project 221c1a5fd69b43b194a4cf34994ab99f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:19:54.610810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:54.612046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-867d4a8b-048f-4f2a-8bbd-49aabeca6ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:19:54.615000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1022/2043] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:19:54 2022] DELETE /identity/v3/projects/221c1a5fd69b43b194a4cf34994ab99f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:54.620271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300822e-8f50-454d-8d62-36d0ddf738ce None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:54.620822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300822e-8f50-454d-8d62-36d0ddf738ce None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:54.620915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300822e-8f50-454d-8d62-36d0ddf738ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:54.643287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2300822e-8f50-454d-8d62-36d0ddf738ce None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:54.644501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2300822e-8f50-454d-8d62-36d0ddf738ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:54.728497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1022/2044] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:54.736928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76437a57-2b24-47d9-b8bd-1f875089aeb2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:54.737320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76437a57-2b24-47d9-b8bd-1f875089aeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:54.738234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76437a57-2b24-47d9-b8bd-1f875089aeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:54.759635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-76437a57-2b24-47d9-b8bd-1f875089aeb2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:54.760899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-76437a57-2b24-47d9-b8bd-1f875089aeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:54.777341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1023/2045] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:19:54.781813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300822e-8f50-454d-8d62-36d0ddf738ce None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:54.792784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300822e-8f50-454d-8d62-36d0ddf738ce None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:54.798088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:54.798842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:54.800080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:54.800507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:54.801520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:19:54.801636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:54.803660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:54.813516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:54.822668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1023/2046] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:19:54.827199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76437a57-2b24-47d9-b8bd-1f875089aeb2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:54.837470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76437a57-2b24-47d9-b8bd-1f875089aeb2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:54.843369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d9228-d9c4-4c77-b332-e3ab08e8ab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:54.844037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d9228-d9c4-4c77-b332-e3ab08e8ab90 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:54.844320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d9228-d9c4-4c77-b332-e3ab08e8ab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:54.844696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d9228-d9c4-4c77-b332-e3ab08e8ab90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:54.845227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d9228-d9c4-4c77-b332-e3ab08e8ab90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:54.847179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d9228-d9c4-4c77-b332-e3ab08e8ab90 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:54.885321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1024/2047] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:54 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:19:54.885535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:54.901782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265a62-b238-4dca-b598-0c12c0ef99e4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:54.969723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35b70a-1446-42ab-aa34-825f825290ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:54.971275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35b70a-1446-42ab-aa34-825f825290ac admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:54.971736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35b70a-1446-42ab-aa34-825f825290ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:54.972216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35b70a-1446-42ab-aa34-825f825290ac admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:54.973039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35b70a-1446-42ab-aa34-825f825290ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:54.975359 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35b70a-1446-42ab-aa34-825f825290ac admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:55.006404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1024/2048] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:54 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:55.007941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d9228-d9c4-4c77-b332-e3ab08e8ab90 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:55.021262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d9228-d9c4-4c77-b332-e3ab08e8ab90 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:55.039785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c14a19-abd4-4291-ae79-e9e4e764ca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:55.040708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c14a19-abd4-4291-ae79-e9e4e764ca1b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:55.041252 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c14a19-abd4-4291-ae79-e9e4e764ca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:55.041681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c14a19-abd4-4291-ae79-e9e4e764ca1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:55.042515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c14a19-abd4-4291-ae79-e9e4e764ca1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:55.044455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c14a19-abd4-4291-ae79-e9e4e764ca1b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:55.078203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35b70a-1446-42ab-aa34-825f825290ac admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:55.092511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1025/2049] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:55.097907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35b70a-1446-42ab-aa34-825f825290ac admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:55.105691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7c63a-aee0-4856-bc3e-5ac56f8b5e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:55.106328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c63a-aee0-4856-bc3e-5ac56f8b5e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:55.106655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c63a-aee0-4856-bc3e-5ac56f8b5e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:55.107022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c63a-aee0-4856-bc3e-5ac56f8b5e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:55.108072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7c63a-aee0-4856-bc3e-5ac56f8b5e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:55.109755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7c63a-aee0-4856-bc3e-5ac56f8b5e72 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:55.133135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c14a19-abd4-4291-ae79-e9e4e764ca1b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:55.133562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1025/2050] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:55.144603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c14a19-abd4-4291-ae79-e9e4e764ca1b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:55.150472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c67831-be8a-44aa-9e69-d2547b41234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:55.151397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c67831-be8a-44aa-9e69-d2547b41234e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:55.151998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c67831-be8a-44aa-9e69-d2547b41234e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:55.152272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c67831-be8a-44aa-9e69-d2547b41234e admin admin] PATH_INFO: `/v3/projects/c0f6ca423a7a43af96b3f743da9c9bff/users/f26bcf3c15024258a5cec3fc8964935c/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:55.178119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c67831-be8a-44aa-9e69-d2547b41234e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0f6ca423a7a43af96b3f743da9c9bff, user_id=f26bcf3c15024258a5cec3fc8964935c, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:55.187538 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c67831-be8a-44aa-9e69-d2547b41234e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:55.209796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1026/2051] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:55 2022] PUT /identity/v3/projects/c0f6ca423a7a43af96b3f743da9c9bff/users/f26bcf3c15024258a5cec3fc8964935c/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:55.221480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4fd1a-316d-4301-b01f-0994309b526f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:55.222393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4fd1a-316d-4301-b01f-0994309b526f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:55.222393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4fd1a-316d-4301-b01f-0994309b526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:55.283827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-60b4fd1a-316d-4301-b01f-0994309b526f None None] MFA Rules not processed for user `f26bcf3c15024258a5cec3fc8964935c`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:55.287521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-60b4fd1a-316d-4301-b01f-0994309b526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:55.403823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1026/2052] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:55.408207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c67831-be8a-44aa-9e69-d2547b41234e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:55.429691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:55.557861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c67831-be8a-44aa-9e69-d2547b41234e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:55.564506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc809c-ad04-44f0-8468-c947b2891c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:55.565231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc809c-ad04-44f0-8468-c947b2891c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:55.565584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc809c-ad04-44f0-8468-c947b2891c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:55.565900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc809c-ad04-44f0-8468-c947b2891c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:55.567218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc809c-ad04-44f0-8468-c947b2891c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:55.569391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc809c-ad04-44f0-8468-c947b2891c7c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:55.621241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1027/2053] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:19:55 2022] POST /identity/v3/projects => generated 399 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:55.626257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4fd1a-316d-4301-b01f-0994309b526f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:55.652898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4fd1a-316d-4301-b01f-0994309b526f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:55.733005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048fed7-12c4-40ec-bfc4-6659bfcfbfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:55.735361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048fed7-12c4-40ec-bfc4-6659bfcfbfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:55.735361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048fed7-12c4-40ec-bfc4-6659bfcfbfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:55.735361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048fed7-12c4-40ec-bfc4-6659bfcfbfd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:55.740243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048fed7-12c4-40ec-bfc4-6659bfcfbfd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:55.744786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048fed7-12c4-40ec-bfc4-6659bfcfbfd0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:55.764654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:55.793732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc809c-ad04-44f0-8468-c947b2891c7c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:55.803647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1027/2054] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:19:55 2022] POST /identity/v3/users => generated 306 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:19:55.806998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc809c-ad04-44f0-8468-c947b2891c7c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:55.815853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9635773-6a38-41d3-b8f8-782f9c981e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:55.819264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9635773-6a38-41d3-b8f8-782f9c981e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:55.819264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9635773-6a38-41d3-b8f8-782f9c981e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:55.819264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9635773-6a38-41d3-b8f8-782f9c981e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:55.819264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9635773-6a38-41d3-b8f8-782f9c981e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:55.822719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9635773-6a38-41d3-b8f8-782f9c981e26 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:55.850453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048fed7-12c4-40ec-bfc4-6659bfcfbfd0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:55.857954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1028/2055] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:55.874883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048fed7-12c4-40ec-bfc4-6659bfcfbfd0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:55.884168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d99e-57cc-4cb3-94f3-9f954ff2dda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:55.884841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d99e-57cc-4cb3-94f3-9f954ff2dda8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:55.885469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d99e-57cc-4cb3-94f3-9f954ff2dda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:55.885727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d99e-57cc-4cb3-94f3-9f954ff2dda8 admin admin] PATH_INFO: `/v3/projects/5d1c52f7fa014bcca35a21ce24c4752a/users/a663258084264caa80cdfe7ad1c7e421/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:55.917959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04d99e-57cc-4cb3-94f3-9f954ff2dda8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1c52f7fa014bcca35a21ce24c4752a, user_id=a663258084264caa80cdfe7ad1c7e421, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:55.922482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04d99e-57cc-4cb3-94f3-9f954ff2dda8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:55.948845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1028/2056] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:55 2022] PUT /identity/v3/projects/5d1c52f7fa014bcca35a21ce24c4752a/users/a663258084264caa80cdfe7ad1c7e421/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:55.950017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9635773-6a38-41d3-b8f8-782f9c981e26 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:55.965981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9635773-6a38-41d3-b8f8-782f9c981e26 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:56.124358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56673e-bafb-4d7e-ae16-ce80d32edb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:56.124914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56673e-bafb-4d7e-ae16-ce80d32edb2c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:56.125198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56673e-bafb-4d7e-ae16-ce80d32edb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:56.125471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56673e-bafb-4d7e-ae16-ce80d32edb2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:56.126137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56673e-bafb-4d7e-ae16-ce80d32edb2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:56.127822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56673e-bafb-4d7e-ae16-ce80d32edb2c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:56.152167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1029/2057] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:19:56.153502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d99e-57cc-4cb3-94f3-9f954ff2dda8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:56.177183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d99e-57cc-4cb3-94f3-9f954ff2dda8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:56.208752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:56.209588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:56.209895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:56.210293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] PATH_INFO: `/v3/projects/5d1c52f7fa014bcca35a21ce24c4752a/users/a663258084264caa80cdfe7ad1c7e421/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:56.264214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1c52f7fa014bcca35a21ce24c4752a, user_id=a663258084264caa80cdfe7ad1c7e421, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:56.266895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:56.321750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1029/2058] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:19:56 2022] PUT /identity/v3/projects/5d1c52f7fa014bcca35a21ce24c4752a/users/a663258084264caa80cdfe7ad1c7e421/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:19:56.334976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8418e55-0ea9-4f24-b35c-57f77993695a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:56.337387 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8418e55-0ea9-4f24-b35c-57f77993695a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:56.337769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8418e55-0ea9-4f24-b35c-57f77993695a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:56.383134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a8418e55-0ea9-4f24-b35c-57f77993695a None None] MFA Rules not processed for user `a663258084264caa80cdfe7ad1c7e421`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:19:56.384921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a8418e55-0ea9-4f24-b35c-57f77993695a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:56.555733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1030/2059] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:19:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:19:56.613394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:56.617174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:19:56.769205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62b6fe-f83f-4fa1-8729-112bc8f76bb7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:56.774299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd69e3b-c606-4fe4-afe9-b50a66122cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:56.774995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd69e3b-c606-4fe4-afe9-b50a66122cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:56.775303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd69e3b-c606-4fe4-afe9-b50a66122cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:56.775647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd69e3b-c606-4fe4-afe9-b50a66122cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:56.807990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd69e3b-c606-4fe4-afe9-b50a66122cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:56.810530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd69e3b-c606-4fe4-afe9-b50a66122cd2 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:56.838192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1030/2060] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:19:56.967188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8418e55-0ea9-4f24-b35c-57f77993695a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:19:56.976133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:19:56.984431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8418e55-0ea9-4f24-b35c-57f77993695a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:19:56.989150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:19:56.989762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:19:56.990036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:19:56.990297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:19:57.112290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:19:57.114904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:19:57.129318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1031/2061] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:19:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:00.191094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc77d6b-fbcf-47dd-9d42-df387fd38a74 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:00.191607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc77d6b-fbcf-47dd-9d42-df387fd38a74 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:00.191684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc77d6b-fbcf-47dd-9d42-df387fd38a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:00.234270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5bc77d6b-fbcf-47dd-9d42-df387fd38a74 None None] MFA Rules not processed for user `5b3928d08cfd420280c1eb7bd1e97832`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:00.236440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5bc77d6b-fbcf-47dd-9d42-df387fd38a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:00.340548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1031/2062] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:00.354058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:00.355863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:00.457851 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce782be-079e-401e-908d-29c3d17e4eba service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:00.463571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:00.464205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:00.464485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:00.464855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:00.468475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:00.598470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:00.599975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:00.623272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1032/2063] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:00.788875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1ab7d-8e9d-4864-9044-8ea66223ed82 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:00.789168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1ab7d-8e9d-4864-9044-8ea66223ed82 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:00.789417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1ab7d-8e9d-4864-9044-8ea66223ed82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:00.818404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c8e1ab7d-8e9d-4864-9044-8ea66223ed82 None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:00.819886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8e1ab7d-8e9d-4864-9044-8ea66223ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:00.904891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:00.928152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1032/2064] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:20:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:00.987926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:00.990295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:01.091792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:01.112423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979b04f-ba34-43ab-b5f5-3e040fd07e83 service glance] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:01.119401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac707c-8b9a-4953-8f70-6d1d7b6ab523 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:01.120248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac707c-8b9a-4953-8f70-6d1d7b6ab523 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:01.120663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac707c-8b9a-4953-8f70-6d1d7b6ab523 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:01.121092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac707c-8b9a-4953-8f70-6d1d7b6ab523 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:01.133126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac707c-8b9a-4953-8f70-6d1d7b6ab523 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:01.135165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac707c-8b9a-4953-8f70-6d1d7b6ab523 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:01.149176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1033/2065] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:20:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:20:02.246531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1ab7d-8e9d-4864-9044-8ea66223ed82 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:02.390928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1ab7d-8e9d-4864-9044-8ea66223ed82 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:02.398659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:02.399381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:02.399664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:02.399938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] PATH_INFO: `/v3/users/f26bcf3c15024258a5cec3fc8964935c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:02.401628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26bcf3c15024258a5cec3fc8964935c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:02.404506 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:02.528075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26bcf3c15024258a5cec3fc8964935c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:02.528804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26bcf3c15024258a5cec3fc8964935c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:02.544083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26bcf3c15024258a5cec3fc8964935c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:02.545134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26bcf3c15024258a5cec3fc8964935c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:02.585648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26bcf3c15024258a5cec3fc8964935c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:02.621002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1033/2066] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:02 2022] DELETE /identity/v3/users/f26bcf3c15024258a5cec3fc8964935c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:02.625361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531cfa2-b29b-4abb-9fa5-32d9bdde2103 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:02.626288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531cfa2-b29b-4abb-9fa5-32d9bdde2103 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:02.626720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531cfa2-b29b-4abb-9fa5-32d9bdde2103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:02.659128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0531cfa2-b29b-4abb-9fa5-32d9bdde2103 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:02.662658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0531cfa2-b29b-4abb-9fa5-32d9bdde2103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:02.811589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1034/2067] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:02.823287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:02.824279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:02.976842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d746fd3-54df-4436-8361-5a47fa3518d6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:02.985961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241da6c-2f23-4d65-835d-3aa6f294887e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:02.991741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241da6c-2f23-4d65-835d-3aa6f294887e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:02.992554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241da6c-2f23-4d65-835d-3aa6f294887e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:02.993095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241da6c-2f23-4d65-835d-3aa6f294887e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:03.008812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241da6c-2f23-4d65-835d-3aa6f294887e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:03.010491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241da6c-2f23-4d65-835d-3aa6f294887e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:03.029935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1034/2068] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:03.930212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531cfa2-b29b-4abb-9fa5-32d9bdde2103 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:03.932725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0531cfa2-b29b-4abb-9fa5-32d9bdde2103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:03.964996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531cfa2-b29b-4abb-9fa5-32d9bdde2103 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:03.986408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:03.987384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:03.987676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:03.987949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] PATH_INFO: `/v3/projects/c0f6ca423a7a43af96b3f743da9c9bff` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:03.990084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0f6ca423a7a43af96b3f743da9c9bff)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:03.992356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:04.133503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0f6ca423a7a43af96b3f743da9c9bff', 'request_id': 'req-e90bdac2-aeac-4034-9973-ba307017a281'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.140733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0f6ca423a7a43af96b3f743da9c9bff', 'request_id': 'req-e90bdac2-aeac-4034-9973-ba307017a281'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.152386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0f6ca423a7a43af96b3f743da9c9bff', 'request_id': 'req-e90bdac2-aeac-4034-9973-ba307017a281'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.173875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] The token cache is being invalidate because project c0f6ca423a7a43af96b3f743da9c9bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:04.174169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.174942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.176753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1035/2069] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:03 2022] DELETE /identity/v3/projects/c0f6ca423a7a43af96b3f743da9c9bff => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:04.180642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241da6c-2f23-4d65-835d-3aa6f294887e service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:04.235586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241da6c-2f23-4d65-835d-3aa6f294887e service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:04.286954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:04.421233 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:04.421905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:04.422183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:04.422771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] PATH_INFO: `/v3/users/a663258084264caa80cdfe7ad1c7e421` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:04.428567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a663258084264caa80cdfe7ad1c7e421)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:04.430514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:04.552548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a663258084264caa80cdfe7ad1c7e421'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.553507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a663258084264caa80cdfe7ad1c7e421'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.574241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a663258084264caa80cdfe7ad1c7e421'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.578526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a663258084264caa80cdfe7ad1c7e421'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.608839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-814449e9-0d77-4668-91b4-a40a9c426347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a663258084264caa80cdfe7ad1c7e421'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:04.631632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1035/2070] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:04 2022] DELETE /identity/v3/users/a663258084264caa80cdfe7ad1c7e421 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:05.680256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:05.682434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:05.700424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bdac2-aeac-4034-9973-ba307017a281 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:05.861134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:05.862156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:05.862464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:05.862905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] PATH_INFO: `/v3/projects/5d1c52f7fa014bcca35a21ce24c4752a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:05.865713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d1c52f7fa014bcca35a21ce24c4752a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:05.868458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:05.913640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:06.049863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1c52f7fa014bcca35a21ce24c4752a', 'request_id': 'req-252a2e5c-553b-4140-9f82-29127ebec2da'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:06.075952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1c52f7fa014bcca35a21ce24c4752a', 'request_id': 'req-252a2e5c-553b-4140-9f82-29127ebec2da'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:06.098434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1c52f7fa014bcca35a21ce24c4752a', 'request_id': 'req-252a2e5c-553b-4140-9f82-29127ebec2da'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:06.136132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] The token cache is being invalidate because project 5d1c52f7fa014bcca35a21ce24c4752a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:06.136419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:06.138737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-252a2e5c-553b-4140-9f82-29127ebec2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:06.141035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1036/2071] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:05 2022] DELETE /identity/v3/projects/5d1c52f7fa014bcca35a21ce24c4752a => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:06.146641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351cc87-f72c-49e7-9655-43ec57ebbfd6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:06.146641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351cc87-f72c-49e7-9655-43ec57ebbfd6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:06.147324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351cc87-f72c-49e7-9655-43ec57ebbfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:06.201313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a351cc87-f72c-49e7-9655-43ec57ebbfd6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:06.202565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a351cc87-f72c-49e7-9655-43ec57ebbfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:06.365542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1036/2072] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:06.376619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882c03-370f-4cca-a974-a9035a921b4f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:06.376902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882c03-370f-4cca-a974-a9035a921b4f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:06.377180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882c03-370f-4cca-a974-a9035a921b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:06.419976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-15882c03-370f-4cca-a974-a9035a921b4f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:06.421216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-15882c03-370f-4cca-a974-a9035a921b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:06.439971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1037/2073] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:06.444411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351cc87-f72c-49e7-9655-43ec57ebbfd6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:06.456965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351cc87-f72c-49e7-9655-43ec57ebbfd6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:06.509026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:06.509662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:06.510006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:06.510326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:06.513205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:06.513713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:06.515602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:06.537633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1037/2074] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:20:06.542433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882c03-370f-4cca-a974-a9035a921b4f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:06.562086 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882c03-370f-4cca-a974-a9035a921b4f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:06.571522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:06.575169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:06.575169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:06.575169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:06.575169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:06.581757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:06.664598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1038/2075] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:06 2022] POST /identity/v3/projects => generated 409 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:06.665841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:06.683748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd35137-8ebb-4e87-9c1d-b40b8f734450 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:06.719882 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:06.821707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:06.826916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:06.843876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940b82d-a951-4654-92a7-4e07c270fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:06.844546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b82d-a951-4654-92a7-4e07c270fbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:06.845372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b82d-a951-4654-92a7-4e07c270fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:06.845372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b82d-a951-4654-92a7-4e07c270fbd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:06.845787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940b82d-a951-4654-92a7-4e07c270fbd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:06.859283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940b82d-a951-4654-92a7-4e07c270fbd6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:06.899886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1038/2076] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:06 2022] POST /identity/v3/users => generated 305 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:20:06.904941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940b82d-a951-4654-92a7-4e07c270fbd6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:06.923313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940b82d-a951-4654-92a7-4e07c270fbd6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:06.936594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1720c-fb3e-4ff5-9da6-9d5f00ae3a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:06.937278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1720c-fb3e-4ff5-9da6-9d5f00ae3a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:06.937565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1720c-fb3e-4ff5-9da6-9d5f00ae3a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:06.937841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1720c-fb3e-4ff5-9da6-9d5f00ae3a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:06.938572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1720c-fb3e-4ff5-9da6-9d5f00ae3a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:06.942908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1720c-fb3e-4ff5-9da6-9d5f00ae3a1c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:06.968753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1039/2077] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:06.973636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1720c-fb3e-4ff5-9da6-9d5f00ae3a1c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:06.975554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc1c15-e88d-49d9-8efd-5ed74820239e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:06.980430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:06.981396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:06.981823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:06.983066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] PATH_INFO: `/v3/users/cace97d1adb44009acf2b7b093d1e885` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:06.984555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cace97d1adb44009acf2b7b093d1e885)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:06.987482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:07.010757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1720c-fb3e-4ff5-9da6-9d5f00ae3a1c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:07.032344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847e3ab-f362-41f0-a67a-25ff8b39710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:07.033075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847e3ab-f362-41f0-a67a-25ff8b39710a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:07.034145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847e3ab-f362-41f0-a67a-25ff8b39710a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:07.034625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847e3ab-f362-41f0-a67a-25ff8b39710a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:07.035815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847e3ab-f362-41f0-a67a-25ff8b39710a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:07.041134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847e3ab-f362-41f0-a67a-25ff8b39710a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:07.089540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1040/2078] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:07.093303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847e3ab-f362-41f0-a67a-25ff8b39710a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:07.099485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cace97d1adb44009acf2b7b093d1e885'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.101324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cace97d1adb44009acf2b7b093d1e885'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.118026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847e3ab-f362-41f0-a67a-25ff8b39710a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:07.120091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cace97d1adb44009acf2b7b093d1e885'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.130349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:07.133454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cace97d1adb44009acf2b7b093d1e885'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.138629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:07.140971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:07.141292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:07.141503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] PATH_INFO: `/v3/projects/899b2b78cdf747eab14ef3b59caae6d2/users/8b3efa640f2a4249a7e34b1c6811343c/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:07.150956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-74e0e4e8-e5ba-4bf5-b951-6d3fc7a5e3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cace97d1adb44009acf2b7b093d1e885'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.163657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=899b2b78cdf747eab14ef3b59caae6d2, user_id=8b3efa640f2a4249a7e34b1c6811343c, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:07.171338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1039/2079] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:06 2022] DELETE /identity/v3/users/cace97d1adb44009acf2b7b093d1e885 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:07.175494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:07.209250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1041/2080] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:07 2022] PUT /identity/v3/projects/899b2b78cdf747eab14ef3b59caae6d2/users/8b3efa640f2a4249a7e34b1c6811343c/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:07.210885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc744d2-9049-4e20-8b72-f7f3e68ea66c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:07.211464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc744d2-9049-4e20-8b72-f7f3e68ea66c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:07.211604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc744d2-9049-4e20-8b72-f7f3e68ea66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:07.281852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7fc744d2-9049-4e20-8b72-f7f3e68ea66c None None] MFA Rules not processed for user `8b3efa640f2a4249a7e34b1c6811343c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:07.283544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7fc744d2-9049-4e20-8b72-f7f3e68ea66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:07.467598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1040/2081] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:07 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:07.473122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:07.475174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:07.561711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc744d2-9049-4e20-8b72-f7f3e68ea66c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:07.563830 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7fc744d2-9049-4e20-8b72-f7f3e68ea66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:07.665098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a27f2-d393-4c30-8141-32550f2f0aae admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:07.675002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b5bb-2985-4b88-a32c-ca49a2fd1b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:07.675827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b5bb-2985-4b88-a32c-ca49a2fd1b41 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:07.676256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b5bb-2985-4b88-a32c-ca49a2fd1b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:07.676546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b5bb-2985-4b88-a32c-ca49a2fd1b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:07.679754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b5bb-2985-4b88-a32c-ca49a2fd1b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:07.682405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b5bb-2985-4b88-a32c-ca49a2fd1b41 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:07.714653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc744d2-9049-4e20-8b72-f7f3e68ea66c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:07.726653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:07.727749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:07.728167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:07.728720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] PATH_INFO: `/v3/projects/15cb4f1fac4f4d728094c3c152c1382d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:07.730773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15cb4f1fac4f4d728094c3c152c1382d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:07.731110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1042/2082] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:07 2022] POST /identity/v3/projects => generated 405 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:07.735276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:07.738411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b5bb-2985-4b88-a32c-ca49a2fd1b41 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:07.895352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15cb4f1fac4f4d728094c3c152c1382d', 'request_id': 'req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.903516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:07.920261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15cb4f1fac4f4d728094c3c152c1382d', 'request_id': 'req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.933090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b5bb-2985-4b88-a32c-ca49a2fd1b41 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:07.946264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15cb4f1fac4f4d728094c3c152c1382d', 'request_id': 'req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.969555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] The token cache is being invalidate because project 15cb4f1fac4f4d728094c3c152c1382d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:07.970019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.970974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9dc4bac6-2041-42a2-b8a0-1a7b945e58c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:07.973119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1041/2083] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:07 2022] DELETE /identity/v3/projects/15cb4f1fac4f4d728094c3c152c1382d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:07.981289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bb838-fbb8-4a58-96f2-4957c2def074 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:07.981802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bb838-fbb8-4a58-96f2-4957c2def074 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:07.982172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bb838-fbb8-4a58-96f2-4957c2def074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.033026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4f7bb838-fbb8-4a58-96f2-4957c2def074 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:08.034419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4f7bb838-fbb8-4a58-96f2-4957c2def074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:08.040315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:08.042779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.043563 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.043841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.044112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.044759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.047841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.089559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1043/2084] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:07 2022] POST /identity/v3/users => generated 309 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:08.090725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.097043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:08.120345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5932e1-bd13-4288-ae45-378200e115a9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.128462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa865ef5-0665-4c7e-8325-ae2ec4a6243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.129257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa865ef5-0665-4c7e-8325-ae2ec4a6243a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.129558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa865ef5-0665-4c7e-8325-ae2ec4a6243a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.129830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa865ef5-0665-4c7e-8325-ae2ec4a6243a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.131469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa865ef5-0665-4c7e-8325-ae2ec4a6243a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.133162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa865ef5-0665-4c7e-8325-ae2ec4a6243a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.149949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1044/2085] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:08.155535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa865ef5-0665-4c7e-8325-ae2ec4a6243a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.171310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa865ef5-0665-4c7e-8325-ae2ec4a6243a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.184633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc1d3e-d602-4452-9b1a-cba17293c2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.186739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1d3e-d602-4452-9b1a-cba17293c2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.187496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1d3e-d602-4452-9b1a-cba17293c2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.188281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc1d3e-d602-4452-9b1a-cba17293c2f7 admin admin] PATH_INFO: `/v3/projects/524281ace7774443aeeed132bd6fa561/users/daaed7a2d5f94550bdd261037c211d97/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.202799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1042/2086] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:08.215612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3006a1-5f56-445d-9221-e97395761677 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.215612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3006a1-5f56-445d-9221-e97395761677 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.215612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3006a1-5f56-445d-9221-e97395761677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.259967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5c3006a1-5f56-445d-9221-e97395761677 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:08.262009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5c3006a1-5f56-445d-9221-e97395761677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:08.269903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc1d3e-d602-4452-9b1a-cba17293c2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=524281ace7774443aeeed132bd6fa561, user_id=daaed7a2d5f94550bdd261037c211d97, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.272619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc1d3e-d602-4452-9b1a-cba17293c2f7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.278406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1043/2087] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:08.279666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3006a1-5f56-445d-9221-e97395761677 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.293792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3006a1-5f56-445d-9221-e97395761677 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.312596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1045/2088] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:08 2022] PUT /identity/v3/projects/524281ace7774443aeeed132bd6fa561/users/daaed7a2d5f94550bdd261037c211d97/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:08.321165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc1d3e-d602-4452-9b1a-cba17293c2f7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.451124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.452057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.452439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.452718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.453371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:08.453740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.455695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.478191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc1d3e-d602-4452-9b1a-cba17293c2f7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.487277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1044/2089] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:08.492432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.499889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29477af-9f8f-4312-b9ce-7eb7ba30950d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.500548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29477af-9f8f-4312-b9ce-7eb7ba30950d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.500880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29477af-9f8f-4312-b9ce-7eb7ba30950d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.501152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29477af-9f8f-4312-b9ce-7eb7ba30950d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.501802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29477af-9f8f-4312-b9ce-7eb7ba30950d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.503542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29477af-9f8f-4312-b9ce-7eb7ba30950d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.506753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35276f-5977-4356-a2c1-af03b91713e8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.512533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638163d0-662d-4b46-b764-286b3bc4aa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.513271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638163d0-662d-4b46-b764-286b3bc4aa05 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.513655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638163d0-662d-4b46-b764-286b3bc4aa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.513998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638163d0-662d-4b46-b764-286b3bc4aa05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.514507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1046/2090] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:20:08.514868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638163d0-662d-4b46-b764-286b3bc4aa05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.516933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638163d0-662d-4b46-b764-286b3bc4aa05 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.518541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29477af-9f8f-4312-b9ce-7eb7ba30950d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.533154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29477af-9f8f-4312-b9ce-7eb7ba30950d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.540637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fd2e2-abec-475f-8a16-6bbd829c46b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.541570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd2e2-abec-475f-8a16-6bbd829c46b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.541911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd2e2-abec-475f-8a16-6bbd829c46b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.542247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd2e2-abec-475f-8a16-6bbd829c46b9 admin admin] PATH_INFO: `/v3/projects/524281ace7774443aeeed132bd6fa561/users/daaed7a2d5f94550bdd261037c211d97/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.547671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fd2e2-abec-475f-8a16-6bbd829c46b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=524281ace7774443aeeed132bd6fa561, user_id=daaed7a2d5f94550bdd261037c211d97, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.555356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1045/2091] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:08 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:08.556318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638163d0-662d-4b46-b764-286b3bc4aa05 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.557610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fd2e2-abec-475f-8a16-6bbd829c46b9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.576775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1047/2092] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:08 2022] PUT /identity/v3/projects/524281ace7774443aeeed132bd6fa561/users/daaed7a2d5f94550bdd261037c211d97/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:08.588878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1e600-3fc6-47c7-b2f6-998897dec9db None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.589162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1e600-3fc6-47c7-b2f6-998897dec9db None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.589487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1e600-3fc6-47c7-b2f6-998897dec9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.631009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2fa1e600-3fc6-47c7-b2f6-998897dec9db None None] MFA Rules not processed for user `daaed7a2d5f94550bdd261037c211d97`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:08.632280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2fa1e600-3fc6-47c7-b2f6-998897dec9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:08.650119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638163d0-662d-4b46-b764-286b3bc4aa05 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.662690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:08.698179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:08.751348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1048/2093] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:08.764185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1e600-3fc6-47c7-b2f6-998897dec9db None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.766319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2fa1e600-3fc6-47c7-b2f6-998897dec9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:08.789202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f78cdd-3477-43fe-89c1-45e7203168bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.790661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f78cdd-3477-43fe-89c1-45e7203168bb admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.791644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f78cdd-3477-43fe-89c1-45e7203168bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.792123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f78cdd-3477-43fe-89c1-45e7203168bb admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.793652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f78cdd-3477-43fe-89c1-45e7203168bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.797865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f78cdd-3477-43fe-89c1-45e7203168bb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.844326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1e600-3fc6-47c7-b2f6-998897dec9db None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.849134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8325-d64a-402c-996a-4bca51605b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.849736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1046/2094] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:08 2022] POST /identity/v3/users => generated 286 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:08.850516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8325-d64a-402c-996a-4bca51605b68 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.851147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8325-d64a-402c-996a-4bca51605b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.851695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8325-d64a-402c-996a-4bca51605b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.853892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f78cdd-3477-43fe-89c1-45e7203168bb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.874316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f78cdd-3477-43fe-89c1-45e7203168bb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.876500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f8325-d64a-402c-996a-4bca51605b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.878131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f8325-d64a-402c-996a-4bca51605b68 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.893893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209de3c-a69a-469c-9b16-5a2b91c2a4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.894556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209de3c-a69a-469c-9b16-5a2b91c2a4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.894834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209de3c-a69a-469c-9b16-5a2b91c2a4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.895107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209de3c-a69a-469c-9b16-5a2b91c2a4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.895812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209de3c-a69a-469c-9b16-5a2b91c2a4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.897424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209de3c-a69a-469c-9b16-5a2b91c2a4a7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.903540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1049/2095] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:08 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:08.920907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1047/2096] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:20:08.925091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8325-d64a-402c-996a-4bca51605b68 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.940824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8325-d64a-402c-996a-4bca51605b68 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.953472 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8fcec-97f0-4d92-b438-d0365e06b7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.954540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8fcec-97f0-4d92-b438-d0365e06b7ab admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.954861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8fcec-97f0-4d92-b438-d0365e06b7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.955415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8fcec-97f0-4d92-b438-d0365e06b7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:08.955881 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8fcec-97f0-4d92-b438-d0365e06b7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:08.957509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8fcec-97f0-4d92-b438-d0365e06b7ab admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:08.972778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1050/2097] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:08.976805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209de3c-a69a-469c-9b16-5a2b91c2a4a7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:08.988355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209de3c-a69a-469c-9b16-5a2b91c2a4a7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:08.994676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08877909-4a4d-4762-ac8d-8d9fc2e45c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:08.995390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08877909-4a4d-4762-ac8d-8d9fc2e45c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:08.995671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08877909-4a4d-4762-ac8d-8d9fc2e45c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:08.995952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08877909-4a4d-4762-ac8d-8d9fc2e45c98 admin admin] PATH_INFO: `/v3/projects/b4f29b97c8ec446ab236bbe84f410006/users/0cf0baa9bb084953bb328a95ee604f10/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:09.019578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08877909-4a4d-4762-ac8d-8d9fc2e45c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f29b97c8ec446ab236bbe84f410006, user_id=0cf0baa9bb084953bb328a95ee604f10, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:09.022396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08877909-4a4d-4762-ac8d-8d9fc2e45c98 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:09.057367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1048/2098] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:08 2022] PUT /identity/v3/projects/b4f29b97c8ec446ab236bbe84f410006/users/0cf0baa9bb084953bb328a95ee604f10/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:09.068739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff12b-4def-419d-a5d5-03b901d5eff3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:09.069306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff12b-4def-419d-a5d5-03b901d5eff3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:09.069618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff12b-4def-419d-a5d5-03b901d5eff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:09.127710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08877909-4a4d-4762-ac8d-8d9fc2e45c98 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:09.128384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3b7ff12b-4def-419d-a5d5-03b901d5eff3 None None] MFA Rules not processed for user `0cf0baa9bb084953bb328a95ee604f10`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:09.129625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3b7ff12b-4def-419d-a5d5-03b901d5eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:09.215831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08877909-4a4d-4762-ac8d-8d9fc2e45c98 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:09.219302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:09.219926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:09.220200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:09.220468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:09.222693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:09.271263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1051/2099] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:09.278327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ff12b-4def-419d-a5d5-03b901d5eff3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:09.280010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3b7ff12b-4def-419d-a5d5-03b901d5eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:09.351213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:09.368458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:09.370244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:09.395409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1049/2100] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:09 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:09.395712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ff12b-4def-419d-a5d5-03b901d5eff3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:09.409605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf0e0d-7596-4b82-932c-1ecb8b9071bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:09.410292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf0e0d-7596-4b82-932c-1ecb8b9071bb service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:09.410573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf0e0d-7596-4b82-932c-1ecb8b9071bb service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:09.410938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf0e0d-7596-4b82-932c-1ecb8b9071bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:09.422097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf0e0d-7596-4b82-932c-1ecb8b9071bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:09.424844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf0e0d-7596-4b82-932c-1ecb8b9071bb service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:09.439404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1052/2101] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:13.308780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:13.310892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:13.429652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:13.508416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe390d9-a274-44a9-ad8c-b5a11e3d85ca service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:13.515746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:13.516461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:13.516769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:13.517039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] PATH_INFO: `/v3/users/c2a8336f29434204ae878d92d86196bf` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:13.518180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2a8336f29434204ae878d92d86196bf)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:13.520185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:13.602298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2a8336f29434204ae878d92d86196bf'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:13.605685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2a8336f29434204ae878d92d86196bf'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:13.625409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2a8336f29434204ae878d92d86196bf'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:13.633972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2a8336f29434204ae878d92d86196bf'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:13.646369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2a8336f29434204ae878d92d86196bf'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:13.669906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1050/2102] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:13 2022] DELETE /identity/v3/users/c2a8336f29434204ae878d92d86196bf => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:13.675986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb620-ade8-4aac-bd7d-707b7fea9902 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:13.676775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb620-ade8-4aac-bd7d-707b7fea9902 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:13.676775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb620-ade8-4aac-bd7d-707b7fea9902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:13.733940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ef0eb620-ade8-4aac-bd7d-707b7fea9902 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:13.735631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ef0eb620-ade8-4aac-bd7d-707b7fea9902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:13.881124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1053/2103] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:13.888795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:13.890702 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:13.969209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1e245-2579-44ad-b4cd-768777a9e75f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:13.973543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77519240-3acd-4fc8-b3ee-9c2cb33de8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:13.974229 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77519240-3acd-4fc8-b3ee-9c2cb33de8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:13.974519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77519240-3acd-4fc8-b3ee-9c2cb33de8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:13.974827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77519240-3acd-4fc8-b3ee-9c2cb33de8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:13.986210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77519240-3acd-4fc8-b3ee-9c2cb33de8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:13.987626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77519240-3acd-4fc8-b3ee-9c2cb33de8a2 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:14.000114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1051/2104] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:14.429971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0eb620-ade8-4aac-bd7d-707b7fea9902 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:14.431266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ef0eb620-ade8-4aac-bd7d-707b7fea9902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:14.450890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0eb620-ade8-4aac-bd7d-707b7fea9902 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:14.458282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:14.458935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:14.459399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:14.459675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] PATH_INFO: `/v3/projects/6b0a57934c5b43e1b92aafb68796efcb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:14.461061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0a57934c5b43e1b92aafb68796efcb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:14.463045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:14.554326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0a57934c5b43e1b92aafb68796efcb', 'request_id': 'req-66af85be-a403-4606-b99f-55ce67f20090'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:14.570209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0a57934c5b43e1b92aafb68796efcb', 'request_id': 'req-66af85be-a403-4606-b99f-55ce67f20090'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:14.589754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0a57934c5b43e1b92aafb68796efcb', 'request_id': 'req-66af85be-a403-4606-b99f-55ce67f20090'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:14.607052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] The token cache is being invalidate because project 6b0a57934c5b43e1b92aafb68796efcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:14.607586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:14.608324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-66af85be-a403-4606-b99f-55ce67f20090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:14.610464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1054/2105] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:14 2022] DELETE /identity/v3/projects/6b0a57934c5b43e1b92aafb68796efcb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:14.615172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb59a4-2aa9-42c8-a850-fcb06944951e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:14.615773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb59a4-2aa9-42c8-a850-fcb06944951e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:14.615773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb59a4-2aa9-42c8-a850-fcb06944951e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:14.639649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dfeb59a4-2aa9-42c8-a850-fcb06944951e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:14.640755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dfeb59a4-2aa9-42c8-a850-fcb06944951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:14.695695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:14.701767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c210a0c-773f-4723-8d44-d0a2a193f23e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:14.702609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c210a0c-773f-4723-8d44-d0a2a193f23e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:14.702609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c210a0c-773f-4723-8d44-d0a2a193f23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:14.709884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1052/2106] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:14.721178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb167f4-baf2-42f0-8153-68ae8b6036e2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:14.721899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb167f4-baf2-42f0-8153-68ae8b6036e2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:14.722330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb167f4-baf2-42f0-8153-68ae8b6036e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:14.723797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1c210a0c-773f-4723-8d44-d0a2a193f23e None None] MFA Rules not processed for user `0987854672d0472ea9dfdfa2d1900a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:14.725176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1c210a0c-773f-4723-8d44-d0a2a193f23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:14.749122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bfb167f4-baf2-42f0-8153-68ae8b6036e2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:14.750330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bfb167f4-baf2-42f0-8153-68ae8b6036e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:14.759705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1053/2107] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:14.765568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb167f4-baf2-42f0-8153-68ae8b6036e2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:14.781468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb167f4-baf2-42f0-8153-68ae8b6036e2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:14.786914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:14.787745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:14.787961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:14.788221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:14.789122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:14.789321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:14.791276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:14.800946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:14.816602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1054/2108] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:14.816832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1055/2109] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:14.816939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:14.828394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93967d-27b3-40cc-9d63-97ab5b94ce19 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:14.833166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210287c5-1894-4409-825b-06f4765829a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:14.833987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210287c5-1894-4409-825b-06f4765829a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:14.834192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210287c5-1894-4409-825b-06f4765829a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:14.834469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210287c5-1894-4409-825b-06f4765829a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:14.835186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210287c5-1894-4409-825b-06f4765829a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:14.839425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210287c5-1894-4409-825b-06f4765829a5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:14.882671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1055/2110] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:14 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:14.886958 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c210a0c-773f-4723-8d44-d0a2a193f23e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:14.914526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c210a0c-773f-4723-8d44-d0a2a193f23e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:14.997913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c7fcd-0b4c-4bd5-b981-5202939d6dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:14.998545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7fcd-0b4c-4bd5-b981-5202939d6dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:14.998809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7fcd-0b4c-4bd5-b981-5202939d6dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:14.999074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7fcd-0b4c-4bd5-b981-5202939d6dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:14.999755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c7fcd-0b4c-4bd5-b981-5202939d6dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.002050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c7fcd-0b4c-4bd5-b981-5202939d6dd0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.026994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1056/2111] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:14 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:20:15.031799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210287c5-1894-4409-825b-06f4765829a5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.045700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210287c5-1894-4409-825b-06f4765829a5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.051782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbdaa6-cc3c-41e0-91c9-2cef8f3b2e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.052434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbdaa6-cc3c-41e0-91c9-2cef8f3b2e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.052812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbdaa6-cc3c-41e0-91c9-2cef8f3b2e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.053173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbdaa6-cc3c-41e0-91c9-2cef8f3b2e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.053966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbdaa6-cc3c-41e0-91c9-2cef8f3b2e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.056535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbdaa6-cc3c-41e0-91c9-2cef8f3b2e99 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.069500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1056/2112] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:15.073567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c7fcd-0b4c-4bd5-b981-5202939d6dd0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.086191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c7fcd-0b4c-4bd5-b981-5202939d6dd0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.091951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabd441-ae67-4725-a693-f79fee0b07c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.092535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabd441-ae67-4725-a693-f79fee0b07c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.092839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabd441-ae67-4725-a693-f79fee0b07c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.093143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabd441-ae67-4725-a693-f79fee0b07c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.093748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabd441-ae67-4725-a693-f79fee0b07c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.095436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabd441-ae67-4725-a693-f79fee0b07c0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.107529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1057/2113] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:15.112019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbdaa6-cc3c-41e0-91c9-2cef8f3b2e99 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.125419 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbdaa6-cc3c-41e0-91c9-2cef8f3b2e99 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.130209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c25d2-b0da-4d1b-b2c6-36c1799ddb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.130904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c25d2-b0da-4d1b-b2c6-36c1799ddb92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.131185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c25d2-b0da-4d1b-b2c6-36c1799ddb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.131579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c25d2-b0da-4d1b-b2c6-36c1799ddb92 admin admin] PATH_INFO: `/v3/projects/5a8826458b604d8baf2e9971ec0a5439/users/2f99173cc296489f966cefdcbb5d9091/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.149121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c25d2-b0da-4d1b-b2c6-36c1799ddb92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8826458b604d8baf2e9971ec0a5439, user_id=2f99173cc296489f966cefdcbb5d9091, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.151908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c25d2-b0da-4d1b-b2c6-36c1799ddb92 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.166069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1057/2114] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:15 2022] PUT /identity/v3/projects/5a8826458b604d8baf2e9971ec0a5439/users/2f99173cc296489f966cefdcbb5d9091/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:15.178646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082750f-2de8-4f72-83de-904eca9357de None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.179140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082750f-2de8-4f72-83de-904eca9357de None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.180613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082750f-2de8-4f72-83de-904eca9357de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.210227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c082750f-2de8-4f72-83de-904eca9357de None None] MFA Rules not processed for user `2f99173cc296489f966cefdcbb5d9091`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:15.211614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c082750f-2de8-4f72-83de-904eca9357de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:15.281681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1058/2115] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:15.286318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c25d2-b0da-4d1b-b2c6-36c1799ddb92 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.360646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c25d2-b0da-4d1b-b2c6-36c1799ddb92 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.367571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c69c1-312a-4ca0-b1c9-7ebc004fa2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.368174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c69c1-312a-4ca0-b1c9-7ebc004fa2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.368479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c69c1-312a-4ca0-b1c9-7ebc004fa2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.368892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c69c1-312a-4ca0-b1c9-7ebc004fa2c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.369509 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c69c1-312a-4ca0-b1c9-7ebc004fa2c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.371774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c69c1-312a-4ca0-b1c9-7ebc004fa2c8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.411692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1058/2116] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:15 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:15.416713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082750f-2de8-4f72-83de-904eca9357de None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.433476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082750f-2de8-4f72-83de-904eca9357de None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.519337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabe7e1-0546-455a-8930-a18530ed36d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.519337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabe7e1-0546-455a-8930-a18530ed36d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.519337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabe7e1-0546-455a-8930-a18530ed36d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.519337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabe7e1-0546-455a-8930-a18530ed36d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.519856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabe7e1-0546-455a-8930-a18530ed36d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.520365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabe7e1-0546-455a-8930-a18530ed36d0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.532510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:15.554570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1059/2117] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:15 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:15.555443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c69c1-312a-4ca0-b1c9-7ebc004fa2c8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.574210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c69c1-312a-4ca0-b1c9-7ebc004fa2c8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.580784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f8377-ef22-4d8d-bc5d-9eb9b5854f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.581393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f8377-ef22-4d8d-bc5d-9eb9b5854f3e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.581682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f8377-ef22-4d8d-bc5d-9eb9b5854f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.581959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f8377-ef22-4d8d-bc5d-9eb9b5854f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.582759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f8377-ef22-4d8d-bc5d-9eb9b5854f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.584665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f8377-ef22-4d8d-bc5d-9eb9b5854f3e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.597371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1059/2118] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:15.601466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabe7e1-0546-455a-8930-a18530ed36d0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.615497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabe7e1-0546-455a-8930-a18530ed36d0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.621243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d308cd4-9bdc-4021-af98-02eae6720240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.621975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d308cd4-9bdc-4021-af98-02eae6720240 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.622404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d308cd4-9bdc-4021-af98-02eae6720240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.622689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d308cd4-9bdc-4021-af98-02eae6720240 admin admin] PATH_INFO: `/v3/projects/f32c9f21aa51472cbe3fcc7b50093e73/users/9d9f3a8d31024e5da6938c3536045aec/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.641871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d308cd4-9bdc-4021-af98-02eae6720240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32c9f21aa51472cbe3fcc7b50093e73, user_id=9d9f3a8d31024e5da6938c3536045aec, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.644760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d308cd4-9bdc-4021-af98-02eae6720240 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.664617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1060/2119] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:15 2022] PUT /identity/v3/projects/f32c9f21aa51472cbe3fcc7b50093e73/users/9d9f3a8d31024e5da6938c3536045aec/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:15.666539 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f8377-ef22-4d8d-bc5d-9eb9b5854f3e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.691653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f8377-ef22-4d8d-bc5d-9eb9b5854f3e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.766796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:15.782132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa346d6-fd80-466f-812e-85150d6428e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.782789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa346d6-fd80-466f-812e-85150d6428e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.783110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa346d6-fd80-466f-812e-85150d6428e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.783460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa346d6-fd80-466f-812e-85150d6428e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.784188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa346d6-fd80-466f-812e-85150d6428e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.785820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa346d6-fd80-466f-812e-85150d6428e7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.797874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1060/2120] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:15.802418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d308cd4-9bdc-4021-af98-02eae6720240 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:15.815183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d308cd4-9bdc-4021-af98-02eae6720240 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:15.821475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:15.822256 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.822616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.822956 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] PATH_INFO: `/v3/projects/f32c9f21aa51472cbe3fcc7b50093e73/users/9d9f3a8d31024e5da6938c3536045aec/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.828689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32c9f21aa51472cbe3fcc7b50093e73, user_id=9d9f3a8d31024e5da6938c3536045aec, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:15.831593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:15.847797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1061/2121] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:15 2022] PUT /identity/v3/projects/f32c9f21aa51472cbe3fcc7b50093e73/users/9d9f3a8d31024e5da6938c3536045aec/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:15.860357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccdb91-dced-4183-91b6-af2e6b2faa03 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:15.860641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccdb91-dced-4183-91b6-af2e6b2faa03 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:15.860915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccdb91-dced-4183-91b6-af2e6b2faa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:15.895043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-eaccdb91-dced-4183-91b6-af2e6b2faa03 None None] MFA Rules not processed for user `9d9f3a8d31024e5da6938c3536045aec`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:15.896407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eaccdb91-dced-4183-91b6-af2e6b2faa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:16.001196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1061/2122] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:16.013618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:16.015393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:16.080026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29889e2a-4151-4bd8-b71a-39d9a2e35417 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:16.083646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc21b11-a6cc-4a83-97b1-3885eb9838ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:16.084249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc21b11-a6cc-4a83-97b1-3885eb9838ec service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:16.084524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc21b11-a6cc-4a83-97b1-3885eb9838ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:16.084832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc21b11-a6cc-4a83-97b1-3885eb9838ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:16.145167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc21b11-a6cc-4a83-97b1-3885eb9838ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:16.146620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc21b11-a6cc-4a83-97b1-3885eb9838ec service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:16.155806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1062/2123] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:20.258206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccdb91-dced-4183-91b6-af2e6b2faa03 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:20.259395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eaccdb91-dced-4183-91b6-af2e6b2faa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:20.376293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccdb91-dced-4183-91b6-af2e6b2faa03 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:20.382798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:20.383680 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:20.383952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:20.384248 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] PATH_INFO: `/v3/users/0cf0baa9bb084953bb328a95ee604f10` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:20.386482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf0baa9bb084953bb328a95ee604f10)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:20.390216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:20.468521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf0baa9bb084953bb328a95ee604f10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:20.469438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf0baa9bb084953bb328a95ee604f10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:20.487318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf0baa9bb084953bb328a95ee604f10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:20.497981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf0baa9bb084953bb328a95ee604f10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:20.500390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:20.511936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf0baa9bb084953bb328a95ee604f10'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:20.520457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1062/2124] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:20 2022] DELETE /identity/v3/users/0cf0baa9bb084953bb328a95ee604f10 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:20.525158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854a219-6f93-4fc9-a1e9-4eed339fc543 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:20.525701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854a219-6f93-4fc9-a1e9-4eed339fc543 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:20.525701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854a219-6f93-4fc9-a1e9-4eed339fc543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:20.552380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6854a219-6f93-4fc9-a1e9-4eed339fc543 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:20.554031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6854a219-6f93-4fc9-a1e9-4eed339fc543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:20.624633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1063/2125] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:20.635664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:20.637375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:20.707323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dffffe-2f06-43c3-87e6-91239730da0c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:20.711798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0efb3d-8e69-45f3-9bf2-d8aef07a0629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:20.712406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0efb3d-8e69-45f3-9bf2-d8aef07a0629 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:20.712716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0efb3d-8e69-45f3-9bf2-d8aef07a0629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:20.713046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0efb3d-8e69-45f3-9bf2-d8aef07a0629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:20.724725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0efb3d-8e69-45f3-9bf2-d8aef07a0629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:20.726199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0efb3d-8e69-45f3-9bf2-d8aef07a0629 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:20.738337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1063/2126] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:21.050629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854a219-6f93-4fc9-a1e9-4eed339fc543 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.052414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6854a219-6f93-4fc9-a1e9-4eed339fc543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:21.068951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854a219-6f93-4fc9-a1e9-4eed339fc543 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:21.075626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:21.076550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.077079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.077359 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] PATH_INFO: `/v3/projects/b4f29b97c8ec446ab236bbe84f410006` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.079182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f29b97c8ec446ab236bbe84f410006)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:21.082302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:21.085866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:21.146883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f29b97c8ec446ab236bbe84f410006', 'request_id': 'req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:21.154852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f29b97c8ec446ab236bbe84f410006', 'request_id': 'req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:21.164290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f29b97c8ec446ab236bbe84f410006', 'request_id': 'req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:21.176099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] The token cache is being invalidate because project b4f29b97c8ec446ab236bbe84f410006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:21.176385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:21.177066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-86be7dc2-2f3d-468f-a410-c5bb806ae7cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:21.178589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1064/2127] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:21 2022] DELETE /identity/v3/projects/b4f29b97c8ec446ab236bbe84f410006 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:21.183841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e5fd-fb32-497b-a586-d23f3ea77e7b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.184148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e5fd-fb32-497b-a586-d23f3ea77e7b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.184447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e5fd-fb32-497b-a586-d23f3ea77e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.205275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a7a3e5fd-fb32-497b-a586-d23f3ea77e7b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:21.206393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a7a3e5fd-fb32-497b-a586-d23f3ea77e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:21.307855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1064/2128] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:21.327631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd3aa3-38e8-45bc-a203-0542f850c4e7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.327631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd3aa3-38e8-45bc-a203-0542f850c4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.327631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd3aa3-38e8-45bc-a203-0542f850c4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.368012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-39cd3aa3-38e8-45bc-a203-0542f850c4e7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:21.369402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-39cd3aa3-38e8-45bc-a203-0542f850c4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:21.379143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1065/2129] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:21.389233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3e5fd-fb32-497b-a586-d23f3ea77e7b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.404289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3e5fd-fb32-497b-a586-d23f3ea77e7b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:21.411319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8b307-ca99-442a-95d0-cef50280b0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:21.411957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8b307-ca99-442a-95d0-cef50280b0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.412336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8b307-ca99-442a-95d0-cef50280b0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.412607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8b307-ca99-442a-95d0-cef50280b0b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.414005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8b307-ca99-442a-95d0-cef50280b0b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:21.414339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8b307-ca99-442a-95d0-cef50280b0b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:21.416164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8b307-ca99-442a-95d0-cef50280b0b5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:21.435857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1065/2130] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:21.440334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd3aa3-38e8-45bc-a203-0542f850c4e7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.459475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd3aa3-38e8-45bc-a203-0542f850c4e7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:21.466585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063985a-4092-497b-aeee-d21e3caf511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:21.467663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063985a-4092-497b-aeee-d21e3caf511a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.467800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505d1ed-4d65-40ef-baf8-db5420e9efff None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.468052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505d1ed-4d65-40ef-baf8-db5420e9efff None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.468154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063985a-4092-497b-aeee-d21e3caf511a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.468319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505d1ed-4d65-40ef-baf8-db5420e9efff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.468511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063985a-4092-497b-aeee-d21e3caf511a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.469277 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063985a-4092-497b-aeee-d21e3caf511a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:21.471402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063985a-4092-497b-aeee-d21e3caf511a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:21.489449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1505d1ed-4d65-40ef-baf8-db5420e9efff None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:21.490567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1505d1ed-4d65-40ef-baf8-db5420e9efff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:21.513186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1066/2131] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:21 2022] POST /identity/v3/projects => generated 403 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:21.518031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063985a-4092-497b-aeee-d21e3caf511a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.610883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063985a-4092-497b-aeee-d21e3caf511a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:21.618081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2ba4f-cb56-4c89-806d-bd2936f05fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:21.618970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ba4f-cb56-4c89-806d-bd2936f05fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.619497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ba4f-cb56-4c89-806d-bd2936f05fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.619679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ba4f-cb56-4c89-806d-bd2936f05fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.620880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2ba4f-cb56-4c89-806d-bd2936f05fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:21.623214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2ba4f-cb56-4c89-806d-bd2936f05fa5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:21.641093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1066/2132] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:20:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:21.663474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505d1ed-4d65-40ef-baf8-db5420e9efff None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.663474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1505d1ed-4d65-40ef-baf8-db5420e9efff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:21.672200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1067/2133] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:21 2022] POST /identity/v3/users => generated 302 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:21.676558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2ba4f-cb56-4c89-806d-bd2936f05fa5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.694662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2ba4f-cb56-4c89-806d-bd2936f05fa5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:21.705507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6628d5-7b3c-4cd8-a1ca-2496eff9ef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:21.708001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6628d5-7b3c-4cd8-a1ca-2496eff9ef17 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.708566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6628d5-7b3c-4cd8-a1ca-2496eff9ef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.709048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:21.710328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6628d5-7b3c-4cd8-a1ca-2496eff9ef17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.711828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6628d5-7b3c-4cd8-a1ca-2496eff9ef17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:21.714209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6628d5-7b3c-4cd8-a1ca-2496eff9ef17 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:21.737884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1068/2134] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:21.741568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6628d5-7b3c-4cd8-a1ca-2496eff9ef17 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.755290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6628d5-7b3c-4cd8-a1ca-2496eff9ef17 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:21.763276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a478ef2-34df-4ff0-a5f0-b9eaf2e3a3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:21.764227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a478ef2-34df-4ff0-a5f0-b9eaf2e3a3de admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.764478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a478ef2-34df-4ff0-a5f0-b9eaf2e3a3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.764694 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a478ef2-34df-4ff0-a5f0-b9eaf2e3a3de admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.765762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a478ef2-34df-4ff0-a5f0-b9eaf2e3a3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:21.767487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a478ef2-34df-4ff0-a5f0-b9eaf2e3a3de admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:21.786970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1069/2135] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:20:21.788139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505d1ed-4d65-40ef-baf8-db5420e9efff None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:21.792478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a478ef2-34df-4ff0-a5f0-b9eaf2e3a3de admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.808574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a478ef2-34df-4ff0-a5f0-b9eaf2e3a3de admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:21.818500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:21.821052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.821409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.821695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] PATH_INFO: `/v3/projects/c81ca52ebfb94f9fb708b4e02ed62a62/users/e484c578a79b4a468df599651c6cad43/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.862733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81ca52ebfb94f9fb708b4e02ed62a62, user_id=e484c578a79b4a468df599651c6cad43, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:21.866841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:21.898381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4496-f597-482f-bd07-76ca2c0b8b13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:21.899121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4496-f597-482f-bd07-76ca2c0b8b13 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.899485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4496-f597-482f-bd07-76ca2c0b8b13 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.899848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4496-f597-482f-bd07-76ca2c0b8b13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:21.902835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1070/2136] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:21 2022] PUT /identity/v3/projects/c81ca52ebfb94f9fb708b4e02ed62a62/users/e484c578a79b4a468df599651c6cad43/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:21.903883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:21.906080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:21.919754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:21.973119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf4496-f597-482f-bd07-76ca2c0b8b13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:21.974773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf4496-f597-482f-bd07-76ca2c0b8b13 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:21.986987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1067/2137] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:20:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:20:21.988619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6cef-74ba-40e5-b62f-5f3288ad82c1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:21.988871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6cef-74ba-40e5-b62f-5f3288ad82c1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:21.989079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6cef-74ba-40e5-b62f-5f3288ad82c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.013049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be39b8e-6c8e-453c-8208-f1ff57a4a55d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:22.018372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:22.019013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.019314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.019585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] PATH_INFO: `/v3/users/8b3efa640f2a4249a7e34b1c6811343c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.020989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3efa640f2a4249a7e34b1c6811343c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:22.022810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:22.031818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-eeaa6cef-74ba-40e5-b62f-5f3288ad82c1 None None] MFA Rules not processed for user `e484c578a79b4a468df599651c6cad43`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:22.033273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eeaa6cef-74ba-40e5-b62f-5f3288ad82c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:22.101082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3efa640f2a4249a7e34b1c6811343c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.107463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3efa640f2a4249a7e34b1c6811343c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.130112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3efa640f2a4249a7e34b1c6811343c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.131128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3efa640f2a4249a7e34b1c6811343c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.156761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:22.165003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1068/2138] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:22.166698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa6cef-74ba-40e5-b62f-5f3288ad82c1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:22.168923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eeaa6cef-74ba-40e5-b62f-5f3288ad82c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:22.169419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3efa640f2a4249a7e34b1c6811343c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.193110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa6cef-74ba-40e5-b62f-5f3288ad82c1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:22.196459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1071/2139] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:21 2022] DELETE /identity/v3/users/8b3efa640f2a4249a7e34b1c6811343c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:22.196459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:22.202105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:22.302793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:22.303972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.304644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.305342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] PATH_INFO: `/v3/users/2f99173cc296489f966cefdcbb5d9091` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.308579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f99173cc296489f966cefdcbb5d9091)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:22.310665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:22.371917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857cad8-ddf0-476a-ac24-297b1e9701b9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:22.380845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f27de-3cc4-4498-a3b0-d417daaac1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:22.382138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f27de-3cc4-4498-a3b0-d417daaac1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.382567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f27de-3cc4-4498-a3b0-d417daaac1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.383039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f27de-3cc4-4498-a3b0-d417daaac1c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.383998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f27de-3cc4-4498-a3b0-d417daaac1c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:22.386993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f99173cc296489f966cefdcbb5d9091'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.387983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f27de-3cc4-4498-a3b0-d417daaac1c1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:22.388161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f99173cc296489f966cefdcbb5d9091'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.416153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f99173cc296489f966cefdcbb5d9091'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.428285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f99173cc296489f966cefdcbb5d9091'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.443891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f99173cc296489f966cefdcbb5d9091'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:22.450674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1072/2140] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:22 2022] POST /identity/v3/projects => generated 403 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:22.452618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772074af-49bb-49c8-8b76-e411f5445276 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.455447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772074af-49bb-49c8-8b76-e411f5445276 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.460677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1069/2141] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:22 2022] DELETE /identity/v3/users/2f99173cc296489f966cefdcbb5d9091 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:22.460677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:22.460677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:22.463264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772074af-49bb-49c8-8b76-e411f5445276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.509870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-772074af-49bb-49c8-8b76-e411f5445276 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:22.517869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-772074af-49bb-49c8-8b76-e411f5445276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:22.532380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:22.593886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272271a8-a8dc-410d-be01-b6c9a3ce59b6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:22.600104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:22.600905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.601478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.602062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.603203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:22.611091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:22.622884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1073/2142] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:22.627030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5bbb6-7965-4e69-9947-73bb3812e117 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.628773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5bbb6-7965-4e69-9947-73bb3812e117 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.628773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5bbb6-7965-4e69-9947-73bb3812e117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.647701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1070/2143] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:22 2022] POST /identity/v3/users => generated 308 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:22.647701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:22.648600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:22.668771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:22.703142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7fb5bbb6-7965-4e69-9947-73bb3812e117 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:22.705054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7fb5bbb6-7965-4e69-9947-73bb3812e117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:22.744178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1074/2144] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:22.745873 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5bbb6-7965-4e69-9947-73bb3812e117 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:22.766523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0eb0-168f-4595-9673-bbbb01a47ba3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:22.770246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40537bfe-f871-49f3-8d97-b0e987aebf8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:22.771005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40537bfe-f871-49f3-8d97-b0e987aebf8d service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.771203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40537bfe-f871-49f3-8d97-b0e987aebf8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.771487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40537bfe-f871-49f3-8d97-b0e987aebf8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.779817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5bbb6-7965-4e69-9947-73bb3812e117 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:22.783049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40537bfe-f871-49f3-8d97-b0e987aebf8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:22.784699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40537bfe-f871-49f3-8d97-b0e987aebf8d service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:22.795573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b8850-ac0c-4a7a-9c14-209e1451e8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:22.798059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b8850-ac0c-4a7a-9c14-209e1451e8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.798572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b8850-ac0c-4a7a-9c14-209e1451e8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.799187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1071/2145] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:22.799975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b8850-ac0c-4a7a-9c14-209e1451e8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.800426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40537bfe-f871-49f3-8d97-b0e987aebf8d service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:22.803647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b8850-ac0c-4a7a-9c14-209e1451e8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:22.805789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b8850-ac0c-4a7a-9c14-209e1451e8f7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:22.811894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40537bfe-f871-49f3-8d97-b0e987aebf8d service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:22.817356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2450892-8f00-4330-b275-89c62c03ba87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:22.818462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2450892-8f00-4330-b275-89c62c03ba87 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.818923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2450892-8f00-4330-b275-89c62c03ba87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.819495 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2450892-8f00-4330-b275-89c62c03ba87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.832879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2450892-8f00-4330-b275-89c62c03ba87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:22.834588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2450892-8f00-4330-b275-89c62c03ba87 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:22.849449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1075/2146] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:22.850816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1072/2147] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:22.854039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b8850-ac0c-4a7a-9c14-209e1451e8f7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:22.873220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b8850-ac0c-4a7a-9c14-209e1451e8f7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:22.902260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fb3f9-0976-4fbe-8db5-5e8891351bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:22.903195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb3f9-0976-4fbe-8db5-5e8891351bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:22.903606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb3f9-0976-4fbe-8db5-5e8891351bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:22.904080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fb3f9-0976-4fbe-8db5-5e8891351bc8 admin admin] PATH_INFO: `/v3/projects/9879204e6e3e4feaaa99eb43f891a4d3/users/8a515c11ba954e5c999edd67aa5e9a7e/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:22.934817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fb3f9-0976-4fbe-8db5-5e8891351bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9879204e6e3e4feaaa99eb43f891a4d3, user_id=8a515c11ba954e5c999edd67aa5e9a7e, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:22.937896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fb3f9-0976-4fbe-8db5-5e8891351bc8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:22.971387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1076/2148] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:22 2022] PUT /identity/v3/projects/9879204e6e3e4feaaa99eb43f891a4d3/users/8a515c11ba954e5c999edd67aa5e9a7e/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:22.976958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2450892-8f00-4330-b275-89c62c03ba87 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:22.998592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2450892-8f00-4330-b275-89c62c03ba87 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:23.180980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62a22b-3a16-49c3-8034-c1ee81bc50f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:23.181942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62a22b-3a16-49c3-8034-c1ee81bc50f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:23.183089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62a22b-3a16-49c3-8034-c1ee81bc50f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:23.184001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62a22b-3a16-49c3-8034-c1ee81bc50f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:23.184590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62a22b-3a16-49c3-8034-c1ee81bc50f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:23.186736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62a22b-3a16-49c3-8034-c1ee81bc50f3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:23.207694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1073/2149] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:20:23.212029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fb3f9-0976-4fbe-8db5-5e8891351bc8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:23.231973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fb3f9-0976-4fbe-8db5-5e8891351bc8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:23.245884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:23.246585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:23.246858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:23.247130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] PATH_INFO: `/v3/projects/9879204e6e3e4feaaa99eb43f891a4d3/users/8a515c11ba954e5c999edd67aa5e9a7e/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:23.260190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9879204e6e3e4feaaa99eb43f891a4d3, user_id=8a515c11ba954e5c999edd67aa5e9a7e, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:23.264158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:23.294418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1077/2150] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:23 2022] PUT /identity/v3/projects/9879204e6e3e4feaaa99eb43f891a4d3/users/8a515c11ba954e5c999edd67aa5e9a7e/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:23.307636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db9a5f-e5ee-409e-8e93-a646af3a508f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:23.308258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db9a5f-e5ee-409e-8e93-a646af3a508f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:23.308761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db9a5f-e5ee-409e-8e93-a646af3a508f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:23.384370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a2db9a5f-e5ee-409e-8e93-a646af3a508f None None] MFA Rules not processed for user `8a515c11ba954e5c999edd67aa5e9a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:23.388606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a2db9a5f-e5ee-409e-8e93-a646af3a508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:23.584965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1074/2151] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:23.601806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:23.606262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:23.647837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db9a5f-e5ee-409e-8e93-a646af3a508f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:23.653698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a2db9a5f-e5ee-409e-8e93-a646af3a508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:23.705558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:23.747368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf5851-d24b-4481-b166-13bc86945792 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:23.764391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:23.765260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:23.766150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:23.766598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:23.781132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db9a5f-e5ee-409e-8e93-a646af3a508f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:23.788223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:23.789074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:23.789546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:23.790015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] PATH_INFO: `/v3/projects/899b2b78cdf747eab14ef3b59caae6d2` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:23.792685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] RBAC: Authorizing `identity:delete_project(project_id=899b2b78cdf747eab14ef3b59caae6d2)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:23.795911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:23.797751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:23.798429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:23.827254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1078/2152] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:23 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:23.828804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:23.833750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:23.860597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3330db1-19b8-47b5-bcf7-c1653d8c7087 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:23.874849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:23.876103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:23.876897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:23.878885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] PATH_INFO: `/v3/projects/5a8826458b604d8baf2e9971ec0a5439` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:23.882252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8826458b604d8baf2e9971ec0a5439)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:23.885671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:23.893371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '899b2b78cdf747eab14ef3b59caae6d2', 'request_id': 'req-91f997d2-3740-4035-9b5b-36d4395f591b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:23.910459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '899b2b78cdf747eab14ef3b59caae6d2', 'request_id': 'req-91f997d2-3740-4035-9b5b-36d4395f591b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:23.925172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '899b2b78cdf747eab14ef3b59caae6d2', 'request_id': 'req-91f997d2-3740-4035-9b5b-36d4395f591b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:23.951709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] The token cache is being invalidate because project 899b2b78cdf747eab14ef3b59caae6d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:23.952108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:23.952864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:23.955356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1075/2153] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:23 2022] DELETE /identity/v3/projects/899b2b78cdf747eab14ef3b59caae6d2 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:23.960710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:23.962767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:23.993015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8826458b604d8baf2e9971ec0a5439', 'request_id': 'req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.015292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8826458b604d8baf2e9971ec0a5439', 'request_id': 'req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.024997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:24.033246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8826458b604d8baf2e9971ec0a5439', 'request_id': 'req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.070961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] The token cache is being invalidate because project 5a8826458b604d8baf2e9971ec0a5439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:24.071431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.072578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.081263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1079/2154] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:23 2022] DELETE /identity/v3/projects/5a8826458b604d8baf2e9971ec0a5439 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:24.082832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:24.085418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:24.127917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f997d2-3740-4035-9b5b-36d4395f591b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:24.207866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:24.216816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:24.218534 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:24.219031 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:24.219626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] PATH_INFO: `/v3/users/daaed7a2d5f94550bdd261037c211d97` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:24.222589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] RBAC: Authorizing `identity:delete_user(user_id=daaed7a2d5f94550bdd261037c211d97)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:24.225862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:24.275497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80dd47-8e68-4c74-8ec2-15ebfe141ff6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:24.280981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:24.281638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:24.281924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:24.282200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] PATH_INFO: `/v3/users/9d9f3a8d31024e5da6938c3536045aec` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:24.288139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9f3a8d31024e5da6938c3536045aec)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:24.290140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:24.333002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daaed7a2d5f94550bdd261037c211d97'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.335116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daaed7a2d5f94550bdd261037c211d97'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.356233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daaed7a2d5f94550bdd261037c211d97'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.392013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daaed7a2d5f94550bdd261037c211d97'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.408424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9f3a8d31024e5da6938c3536045aec'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.408772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daaed7a2d5f94550bdd261037c211d97'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.411173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9f3a8d31024e5da6938c3536045aec'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.427212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1076/2155] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:23 2022] DELETE /identity/v3/users/daaed7a2d5f94550bdd261037c211d97 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:24.429140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:24.431264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:24.438454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9f3a8d31024e5da6938c3536045aec'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.441764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9f3a8d31024e5da6938c3536045aec'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.494738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03dc8af5-35f7-448e-b906-1d9639dcb42c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9f3a8d31024e5da6938c3536045aec'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.566255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1080/2156] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:24 2022] DELETE /identity/v3/users/9d9f3a8d31024e5da6938c3536045aec => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:24.651933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917970aa-5fec-4448-b3df-953125e4483c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:24.813602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:24.814410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:24.814574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:24.815280 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] PATH_INFO: `/v3/users/e484c578a79b4a468df599651c6cad43` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:24.818730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e484c578a79b4a468df599651c6cad43)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:24.824443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:24.905776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:24.954520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e484c578a79b4a468df599651c6cad43'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.955258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e484c578a79b4a468df599651c6cad43'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.976432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e484c578a79b4a468df599651c6cad43'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:24.989994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e484c578a79b4a468df599651c6cad43'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.000229 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e484c578a79b4a468df599651c6cad43'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.025881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1077/2157] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:24 2022] DELETE /identity/v3/users/e484c578a79b4a468df599651c6cad43 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:25.035516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca1f6-a2a3-478f-ac2e-e1dfc596ccdf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:25.035988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca1f6-a2a3-478f-ac2e-e1dfc596ccdf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:25.036108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca1f6-a2a3-478f-ac2e-e1dfc596ccdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:25.057474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:25.059142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:25.062770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9daca1f6-a2a3-478f-ac2e-e1dfc596ccdf None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:25.064040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9daca1f6-a2a3-478f-ac2e-e1dfc596ccdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:25.184137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1081/2158] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:25.204575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daca1f6-a2a3-478f-ac2e-e1dfc596ccdf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:25.206856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9daca1f6-a2a3-478f-ac2e-e1dfc596ccdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:25.220213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:25.247820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd53911-4ed1-4100-b786-1e359a6394d3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:25.254996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:25.272755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:25.273163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:25.274047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] PATH_INFO: `/v3/projects/524281ace7774443aeeed132bd6fa561` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:25.275190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=524281ace7774443aeeed132bd6fa561)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:25.277272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:25.322601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daca1f6-a2a3-478f-ac2e-e1dfc596ccdf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:25.351941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:25.353050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:25.355584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:25.357237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:25.378669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '524281ace7774443aeeed132bd6fa561', 'request_id': 'req-42c90c49-473f-452a-8923-c43aaad9ed11'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.394890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '524281ace7774443aeeed132bd6fa561', 'request_id': 'req-42c90c49-473f-452a-8923-c43aaad9ed11'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.399638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:25.405428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '524281ace7774443aeeed132bd6fa561', 'request_id': 'req-42c90c49-473f-452a-8923-c43aaad9ed11'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.407508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:25.436507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] The token cache is being invalidate because project 524281ace7774443aeeed132bd6fa561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:25.436727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.437402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42c90c49-473f-452a-8923-c43aaad9ed11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.439199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1078/2159] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:25 2022] DELETE /identity/v3/projects/524281ace7774443aeeed132bd6fa561 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:25.445542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0917ce-45ac-48b2-a045-dcfff6dfc29e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:25.445823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0917ce-45ac-48b2-a045-dcfff6dfc29e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:25.446233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0917ce-45ac-48b2-a045-dcfff6dfc29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:25.453976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1082/2160] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:25.484536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2f0917ce-45ac-48b2-a045-dcfff6dfc29e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:25.485776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2f0917ce-45ac-48b2-a045-dcfff6dfc29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:25.563762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:25.605008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:25.606844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:25.641587 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1079/2161] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:25.646820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7cc30-58f9-4e49-a601-48145e70aaaa service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:25.654139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a7267-6d9e-423b-951c-788f21df3bcd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:25.654468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a7267-6d9e-423b-951c-788f21df3bcd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:25.654752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a7267-6d9e-423b-951c-788f21df3bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:25.706303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:25.719420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:25.719839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:25.720163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] PATH_INFO: `/v3/projects/f32c9f21aa51472cbe3fcc7b50093e73` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:25.720516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f66a7267-6d9e-423b-951c-788f21df3bcd None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:25.721824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f66a7267-6d9e-423b-951c-788f21df3bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:25.728204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f32c9f21aa51472cbe3fcc7b50093e73)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:25.735108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:25.743071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1080/2162] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:25.749600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a7267-6d9e-423b-951c-788f21df3bcd None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:25.796829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a7267-6d9e-423b-951c-788f21df3bcd None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:25.809854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:25.810651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:25.811287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:25.811766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:25.813696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:25.814592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:25.816841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:25.854064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1081/2163] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:25.859373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:25.875291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f32c9f21aa51472cbe3fcc7b50093e73', 'request_id': 'req-43562976-1fe2-4cc3-8697-047938bb901a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.886839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f32c9f21aa51472cbe3fcc7b50093e73', 'request_id': 'req-43562976-1fe2-4cc3-8697-047938bb901a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.905235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f32c9f21aa51472cbe3fcc7b50093e73', 'request_id': 'req-43562976-1fe2-4cc3-8697-047938bb901a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.927386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] The token cache is being invalidate because project f32c9f21aa51472cbe3fcc7b50093e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:25.927386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.928096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-43562976-1fe2-4cc3-8697-047938bb901a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:25.931340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1083/2164] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:25 2022] DELETE /identity/v3/projects/f32c9f21aa51472cbe3fcc7b50093e73 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:25.936142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2cead-0066-4e73-b790-6444dc2c6a27 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:25.936533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2cead-0066-4e73-b790-6444dc2c6a27 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:25.936708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2cead-0066-4e73-b790-6444dc2c6a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:25.940428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e06868-c4b8-4b59-b5c4-d09775f850e6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:25.980388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-62d2cead-0066-4e73-b790-6444dc2c6a27 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:25.983055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-62d2cead-0066-4e73-b790-6444dc2c6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:26.140993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc14c4-7464-4100-9a4c-df3221125957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:26.141555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc14c4-7464-4100-9a4c-df3221125957 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:26.141837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc14c4-7464-4100-9a4c-df3221125957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:26.142110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc14c4-7464-4100-9a4c-df3221125957 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:26.142761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc14c4-7464-4100-9a4c-df3221125957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:26.145368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc14c4-7464-4100-9a4c-df3221125957 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:26.180559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1084/2165] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:26.181920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2cead-0066-4e73-b790-6444dc2c6a27 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:26.184399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-62d2cead-0066-4e73-b790-6444dc2c6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:26.200670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1082/2166] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:25 2022] POST /identity/v3/projects => generated 365 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:26.202452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30177-51f6-4611-9dee-6b13e89c1654 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:26.202877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30177-51f6-4611-9dee-6b13e89c1654 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:26.203380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30177-51f6-4611-9dee-6b13e89c1654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:26.224983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2cead-0066-4e73-b790-6444dc2c6a27 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:26.247333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2cb30177-51f6-4611-9dee-6b13e89c1654 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:26.248672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2cb30177-51f6-4611-9dee-6b13e89c1654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:26.316139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:26.371398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:26.372979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:26.373777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:26.374286 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] PATH_INFO: `/v3/projects/c81ca52ebfb94f9fb708b4e02ed62a62` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:26.376824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81ca52ebfb94f9fb708b4e02ed62a62)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:26.379448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:26.392766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1083/2167] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:26.393746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb30177-51f6-4611-9dee-6b13e89c1654 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:26.397065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2cb30177-51f6-4611-9dee-6b13e89c1654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:26.433310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb30177-51f6-4611-9dee-6b13e89c1654 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:26.453006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81ca52ebfb94f9fb708b4e02ed62a62', 'request_id': 'req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.464757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81ca52ebfb94f9fb708b4e02ed62a62', 'request_id': 'req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.479902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81ca52ebfb94f9fb708b4e02ed62a62', 'request_id': 'req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.502732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] The token cache is being invalidate because project c81ca52ebfb94f9fb708b4e02ed62a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:26.503677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.505096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.507065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1085/2168] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:26 2022] DELETE /identity/v3/projects/c81ca52ebfb94f9fb708b4e02ed62a62 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:26.509208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:26.510485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:26.511713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:26.549499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:26.550277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:26.550601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:26.550835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:26.551513 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:26.557545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:26.630960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1084/2169] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:26 2022] POST /identity/v3/users => generated 283 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:26.631434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:26.636338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:26.656968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c7436-e8be-4e07-8f5a-a0b2f0b054b6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:26.662043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d0b78-2f23-4e97-a0c8-18dcff3af6a2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:26.669216 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:26.670143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:26.670514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:26.671038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:26.672221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:26.672396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:26.672675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:26.673029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:26.673454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:26.673727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] PATH_INFO: `/v3/users/8a515c11ba954e5c999edd67aa5e9a7e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:26.674620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:26.675390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a515c11ba954e5c999edd67aa5e9a7e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:26.677595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:26.691373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1086/2170] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:26.692468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:26.694908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:26.764872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d9d5-a2e5-43d1-9514-a5de42d626c6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:26.777034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31119849-ce13-495a-a3d3-a88f2e878542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:26.778917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119849-ce13-495a-a3d3-a88f2e878542 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:26.779756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119849-ce13-495a-a3d3-a88f2e878542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:26.780484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119849-ce13-495a-a3d3-a88f2e878542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:26.782521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31119849-ce13-495a-a3d3-a88f2e878542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:26.787492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31119849-ce13-495a-a3d3-a88f2e878542 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:26.810787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a515c11ba954e5c999edd67aa5e9a7e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.814100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a515c11ba954e5c999edd67aa5e9a7e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.819696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1087/2171] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:26.820673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31119849-ce13-495a-a3d3-a88f2e878542 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:26.834578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a515c11ba954e5c999edd67aa5e9a7e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.845794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a515c11ba954e5c999edd67aa5e9a7e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.846036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31119849-ce13-495a-a3d3-a88f2e878542 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:26.856393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a515c11ba954e5c999edd67aa5e9a7e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:26.864884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:26.865557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:26.865936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:26.867810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1085/2172] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:26 2022] DELETE /identity/v3/users/8a515c11ba954e5c999edd67aa5e9a7e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:26.868629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:26.869170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:26.870125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:26.870394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:26.872930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:26.947464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1088/2173] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:26 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:26.949097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:26.957758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.004972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:27.036863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb4317-2f84-45e3-b410-5afd5da207d7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.087937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8795362-121c-442f-9d02-5519051f45e0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.094123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.095663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.096164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.096756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.097689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.100180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:27.128942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1089/2174] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:27.130130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:27.136535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.146296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:27.162445 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d6d89-7aaf-47bf-8391-9eb3ab09b110 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.168776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363381d-9adb-41dd-a311-e910824480f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.169404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363381d-9adb-41dd-a311-e910824480f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.169664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363381d-9adb-41dd-a311-e910824480f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.169934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363381d-9adb-41dd-a311-e910824480f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.170604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363381d-9adb-41dd-a311-e910824480f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.173026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363381d-9adb-41dd-a311-e910824480f4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:27.173566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.174652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.174928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.175212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] PATH_INFO: `/v3/users/9f7907cabdd14fdd81d6bddccb471bd4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.177567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f7907cabdd14fdd81d6bddccb471bd4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.185395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:27.221495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1090/2175] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:27 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:27.224081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363381d-9adb-41dd-a311-e910824480f4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:27.246456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363381d-9adb-41dd-a311-e910824480f4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.252443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43a09-46d8-4485-9dc8-0670f6e5049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.253165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43a09-46d8-4485-9dc8-0670f6e5049a admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.253396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43a09-46d8-4485-9dc8-0670f6e5049a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.253736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43a09-46d8-4485-9dc8-0670f6e5049a admin admin] PATH_INFO: `/v3/projects/c94606bf6d564285965b9d0256de535c/users/f8228d72e0d044ed9a78574bbc5d718b/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.289333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf43a09-46d8-4485-9dc8-0670f6e5049a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94606bf6d564285965b9d0256de535c, user_id=f8228d72e0d044ed9a78574bbc5d718b, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.292362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf43a09-46d8-4485-9dc8-0670f6e5049a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:27.293803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f7907cabdd14fdd81d6bddccb471bd4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.294941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f7907cabdd14fdd81d6bddccb471bd4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.332420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1091/2176] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:27 2022] PUT /identity/v3/projects/c94606bf6d564285965b9d0256de535c/users/f8228d72e0d044ed9a78574bbc5d718b/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:27.332774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f7907cabdd14fdd81d6bddccb471bd4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.333463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43a09-46d8-4485-9dc8-0670f6e5049a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:27.343752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f7907cabdd14fdd81d6bddccb471bd4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.367950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-328cad44-16b0-4d88-9cc0-c9a453501df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f7907cabdd14fdd81d6bddccb471bd4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.378495 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1086/2177] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:26 2022] DELETE /identity/v3/users/9f7907cabdd14fdd81d6bddccb471bd4 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:27.383709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85fff-47cb-44bd-80a8-8bc2f24bd572 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.385728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85fff-47cb-44bd-80a8-8bc2f24bd572 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.386679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85fff-47cb-44bd-80a8-8bc2f24bd572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.424847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43a09-46d8-4485-9dc8-0670f6e5049a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.457775 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-45b85fff-47cb-44bd-80a8-8bc2f24bd572 None None] MFA Rules not processed for user `f8228d72e0d044ed9a78574bbc5d718b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:27.459061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-45b85fff-47cb-44bd-80a8-8bc2f24bd572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.506812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fefd0-6d8a-4e8b-815e-2136383cd42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.507147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fefd0-6d8a-4e8b-815e-2136383cd42f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.507914 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fefd0-6d8a-4e8b-815e-2136383cd42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.508199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fefd0-6d8a-4e8b-815e-2136383cd42f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.509667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fefd0-6d8a-4e8b-815e-2136383cd42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.512205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fefd0-6d8a-4e8b-815e-2136383cd42f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:27.547222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1092/2178] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:27.552839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b405-f7ee-4fc3-99e4-5c51ab7f22bc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.553341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b405-f7ee-4fc3-99e4-5c51ab7f22bc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.553665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b405-f7ee-4fc3-99e4-5c51ab7f22bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.630031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7460b405-f7ee-4fc3-99e4-5c51ab7f22bc None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:27.648222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7460b405-f7ee-4fc3-99e4-5c51ab7f22bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.649418 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1087/2179] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:27.650290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85fff-47cb-44bd-80a8-8bc2f24bd572 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:27.654408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-45b85fff-47cb-44bd-80a8-8bc2f24bd572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.659911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1093/2180] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:27.660767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460b405-f7ee-4fc3-99e4-5c51ab7f22bc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:27.668380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7460b405-f7ee-4fc3-99e4-5c51ab7f22bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.685919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460b405-f7ee-4fc3-99e4-5c51ab7f22bc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.691699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.692361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.692644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.692917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] PATH_INFO: `/v3/projects/9879204e6e3e4feaaa99eb43f891a4d3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.694238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9879204e6e3e4feaaa99eb43f891a4d3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.696570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:27.698865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85fff-47cb-44bd-80a8-8bc2f24bd572 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.719451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.719627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:27.721223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.722125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.723041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.726271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.730664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:27.763155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1088/2181] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:27.763155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:27.769348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.781780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9879204e6e3e4feaaa99eb43f891a4d3', 'request_id': 'req-0b2fabda-29db-497f-b974-174c73732218'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.789494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c1cc2-1c10-4e47-aeeb-b4f8423ddc98 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.790385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9879204e6e3e4feaaa99eb43f891a4d3', 'request_id': 'req-0b2fabda-29db-497f-b974-174c73732218'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.807545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9879204e6e3e4feaaa99eb43f891a4d3', 'request_id': 'req-0b2fabda-29db-497f-b974-174c73732218'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.836964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] The token cache is being invalidate because project 9879204e6e3e4feaaa99eb43f891a4d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:27.837948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.839684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:27.846953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1094/2182] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:27 2022] DELETE /identity/v3/projects/9879204e6e3e4feaaa99eb43f891a4d3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:27.846953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:27.846953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.901965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:27.944164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.945286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.945799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.946281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.947317 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.949534 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:27.953825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fabda-29db-497f-b974-174c73732218 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:27.957735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1798c-58cb-4541-a266-2022889b6379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:27.958364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1798c-58cb-4541-a266-2022889b6379 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:27.958642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1798c-58cb-4541-a266-2022889b6379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:27.958917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1798c-58cb-4541-a266-2022889b6379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:27.961011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-17f1798c-58cb-4541-a266-2022889b6379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:27.975289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1798c-58cb-4541-a266-2022889b6379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:27.976933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1798c-58cb-4541-a266-2022889b6379 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:28.003527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1089/2183] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:27 2022] POST /identity/v3/projects => generated 367 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:28.004862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:28.006900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:28.007921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1095/2184] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:28.009401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53dc59-b09a-4f21-a34c-771a75680075 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.009688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53dc59-b09a-4f21-a34c-771a75680075 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.009963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53dc59-b09a-4f21-a34c-771a75680075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.063593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5d53dc59-b09a-4f21-a34c-771a75680075 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:28.065803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5d53dc59-b09a-4f21-a34c-771a75680075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:28.144475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5c904-52fe-4afd-b7b7-d51f0eb9fb2a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:28.171696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd7ed3-6a04-4f9a-9517-6dbe1e033d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:28.172768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd7ed3-6a04-4f9a-9517-6dbe1e033d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.173839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd7ed3-6a04-4f9a-9517-6dbe1e033d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.174332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd7ed3-6a04-4f9a-9517-6dbe1e033d2f admin admin] PATH_INFO: `/v3/projects/a8426753141444c581bd6cecf00b5221/users/ce19853819814ed9906aa9b5cae809cb/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.237188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd7ed3-6a04-4f9a-9517-6dbe1e033d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8426753141444c581bd6cecf00b5221, user_id=ce19853819814ed9906aa9b5cae809cb, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:28.239938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd7ed3-6a04-4f9a-9517-6dbe1e033d2f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:28.269760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1096/2186] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:28.271386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53dc59-b09a-4f21-a34c-771a75680075 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:28.275374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5d53dc59-b09a-4f21-a34c-771a75680075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:28.290563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1090/2186] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:28 2022] PUT /identity/v3/projects/a8426753141444c581bd6cecf00b5221/users/ce19853819814ed9906aa9b5cae809cb/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:28.292425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1986e-97c6-40ec-93f8-8531f0c089c9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.292602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1986e-97c6-40ec-93f8-8531f0c089c9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.292766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1986e-97c6-40ec-93f8-8531f0c089c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.307532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53dc59-b09a-4f21-a34c-771a75680075 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:28.338283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-13a1986e-97c6-40ec-93f8-8531f0c089c9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:28.343772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-13a1986e-97c6-40ec-93f8-8531f0c089c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:28.405761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:28.451305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5c249-20b4-433f-9aa8-531c042b0d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:28.452087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5c249-20b4-433f-9aa8-531c042b0d3e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.452463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5c249-20b4-433f-9aa8-531c042b0d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.452832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5c249-20b4-433f-9aa8-531c042b0d3e admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.455489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5c249-20b4-433f-9aa8-531c042b0d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:28.460847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5c249-20b4-433f-9aa8-531c042b0d3e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:28.509387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1097/2187] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:28 2022] POST /identity/v3/users => generated 290 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:28.516758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809410e-85cb-4b82-931e-272beba4bdc7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.517060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809410e-85cb-4b82-931e-272beba4bdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.517311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809410e-85cb-4b82-931e-272beba4bdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.518660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1091/2188] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:28.519638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1986e-97c6-40ec-93f8-8531f0c089c9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:28.542907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1986e-97c6-40ec-93f8-8531f0c089c9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:28.555290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe75ef-fd11-46ea-a87e-476113a38033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:28.555853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe75ef-fd11-46ea-a87e-476113a38033 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.556135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe75ef-fd11-46ea-a87e-476113a38033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.556410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe75ef-fd11-46ea-a87e-476113a38033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.557131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe75ef-fd11-46ea-a87e-476113a38033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:28.562678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe75ef-fd11-46ea-a87e-476113a38033 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:28.577241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5809410e-85cb-4b82-931e-272beba4bdc7 None None] MFA Rules not processed for user `ce19853819814ed9906aa9b5cae809cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:28.578515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5809410e-85cb-4b82-931e-272beba4bdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:28.587832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1092/2189] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:28.589048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe75ef-fd11-46ea-a87e-476113a38033 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:28.621461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe75ef-fd11-46ea-a87e-476113a38033 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:28.643397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:28.644103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.644388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.644660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.645428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:28.645738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:28.650284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:28.673974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1093/2190] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:28.688317 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:28.693683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402e88f-ca57-458f-81e0-8ec261628ac8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:28.699298 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e6d34-3c8c-4dce-ad81-58303f0d9b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:28.699985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6d34-3c8c-4dce-ad81-58303f0d9b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.700410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6d34-3c8c-4dce-ad81-58303f0d9b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.700687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6d34-3c8c-4dce-ad81-58303f0d9b61 admin admin] PATH_INFO: `/v3/projects/118f7575965d4102b243c7ffc702efc9/users/8034f1ec47824eda9e4862fec718254f/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.721996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e6d34-3c8c-4dce-ad81-58303f0d9b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118f7575965d4102b243c7ffc702efc9, user_id=8034f1ec47824eda9e4862fec718254f, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:28.721996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e6d34-3c8c-4dce-ad81-58303f0d9b61 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:28.752966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1094/2191] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:28 2022] PUT /identity/v3/projects/118f7575965d4102b243c7ffc702efc9/users/8034f1ec47824eda9e4862fec718254f/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:28.759359 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1098/2192] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:28.759688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e6d34-3c8c-4dce-ad81-58303f0d9b61 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:28.765830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809410e-85cb-4b82-931e-272beba4bdc7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:28.765830 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5809410e-85cb-4b82-931e-272beba4bdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:28.779495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809410e-85cb-4b82-931e-272beba4bdc7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:28.819174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:28.916093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:28.917483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:28.917776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:28.918157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] PATH_INFO: `/v3/projects/0d1b440ef8644dfc82577c155ad743da` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:28.919467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1b440ef8644dfc82577c155ad743da)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:28.921747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:28.996200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e6d34-3c8c-4dce-ad81-58303f0d9b61 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.054860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1b440ef8644dfc82577c155ad743da', 'request_id': 'req-50b551d2-802a-422d-813d-5ed954b46324'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:29.066151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1b440ef8644dfc82577c155ad743da', 'request_id': 'req-50b551d2-802a-422d-813d-5ed954b46324'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:29.081021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:29.090570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1b440ef8644dfc82577c155ad743da', 'request_id': 'req-50b551d2-802a-422d-813d-5ed954b46324'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:29.113157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] The token cache is being invalidate because project 0d1b440ef8644dfc82577c155ad743da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:29.113541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:29.114210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:29.116258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1099/2193] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:28 2022] DELETE /identity/v3/projects/0d1b440ef8644dfc82577c155ad743da => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:29.117171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.120724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:29.147737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.148442 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.149288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.149600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.150283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.155483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.194756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1095/2194] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:28 2022] POST /identity/v3/projects => generated 383 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:29.194756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.194756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:29.274024 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b551d2-802a-422d-813d-5ed954b46324 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.285959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b639e-5e25-44e3-8c79-a36405f2b4fe admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.288656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca281a-35eb-4979-90c7-8b216f2f11fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.289292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca281a-35eb-4979-90c7-8b216f2f11fe admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.289567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca281a-35eb-4979-90c7-8b216f2f11fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.289834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca281a-35eb-4979-90c7-8b216f2f11fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.290538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca281a-35eb-4979-90c7-8b216f2f11fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.292502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca281a-35eb-4979-90c7-8b216f2f11fe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.295270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.295985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.296263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.296844 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.297565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.299954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.311966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1100/2195] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:29.313758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eff86-8fc3-4289-9065-33ccab69c3ba None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.314110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eff86-8fc3-4289-9065-33ccab69c3ba None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.314450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eff86-8fc3-4289-9065-33ccab69c3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.336919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-342eff86-8fc3-4289-9065-33ccab69c3ba None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:29.338134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-342eff86-8fc3-4289-9065-33ccab69c3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:29.345477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1096/2196] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:29 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:29.346205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.349041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1101/2197] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:29.350176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eff86-8fc3-4289-9065-33ccab69c3ba None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.350365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:29.363177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eff86-8fc3-4289-9065-33ccab69c3ba None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.393480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:29.456481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878616af-38c5-4925-b8b6-49ec4019eef7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.459111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9cdd7-d586-41c6-9bc0-a39fcfc3cbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.459865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9cdd7-d586-41c6-9bc0-a39fcfc3cbf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.460322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9cdd7-d586-41c6-9bc0-a39fcfc3cbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.460452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9cdd7-d586-41c6-9bc0-a39fcfc3cbf3 admin admin] PATH_INFO: `/v3/projects/118f7575965d4102b243c7ffc702efc9/users/8034f1ec47824eda9e4862fec718254f/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.461834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0b75c-bf5e-4f0e-802b-cdd77073e495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.462691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0b75c-bf5e-4f0e-802b-cdd77073e495 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.462834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9cdd7-d586-41c6-9bc0-a39fcfc3cbf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118f7575965d4102b243c7ffc702efc9, user_id=8034f1ec47824eda9e4862fec718254f, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.462931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0b75c-bf5e-4f0e-802b-cdd77073e495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.463254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0b75c-bf5e-4f0e-802b-cdd77073e495 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.464552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0b75c-bf5e-4f0e-802b-cdd77073e495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.465754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9cdd7-d586-41c6-9bc0-a39fcfc3cbf3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.466175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0b75c-bf5e-4f0e-802b-cdd77073e495 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.483644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1102/2198] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:29 2022] PUT /identity/v3/projects/118f7575965d4102b243c7ffc702efc9/users/8034f1ec47824eda9e4862fec718254f/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:29.484910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9cdd7-d586-41c6-9bc0-a39fcfc3cbf3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.503374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1097/2199] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:29 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:29.504914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf3a1b-cd9f-4091-9067-a2b3cfceccbd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.505199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf3a1b-cd9f-4091-9067-a2b3cfceccbd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.505469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf3a1b-cd9f-4091-9067-a2b3cfceccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.536013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-66cf3a1b-cd9f-4091-9067-a2b3cfceccbd None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:29.537384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66cf3a1b-cd9f-4091-9067-a2b3cfceccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:29.590945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9cdd7-d586-41c6-9bc0-a39fcfc3cbf3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.598157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cead47-8abe-4940-a34f-594d937ea1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.598778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cead47-8abe-4940-a34f-594d937ea1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.599058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cead47-8abe-4940-a34f-594d937ea1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.599358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cead47-8abe-4940-a34f-594d937ea1d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.599996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cead47-8abe-4940-a34f-594d937ea1d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.602547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cead47-8abe-4940-a34f-594d937ea1d1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.622669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1098/2200] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:29.624123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:29.624433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e706fd4-1e0c-461f-9428-51817581a3eb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.624779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e706fd4-1e0c-461f-9428-51817581a3eb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.625267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e706fd4-1e0c-461f-9428-51817581a3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.634528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1103/2201] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:29 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:29.637767 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cead47-8abe-4940-a34f-594d937ea1d1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.651312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cead47-8abe-4940-a34f-594d937ea1d1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.656573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce4486-6f56-47d7-abca-885eb642630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.657287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4486-6f56-47d7-abca-885eb642630c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.657534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4486-6f56-47d7-abca-885eb642630c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.657815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4486-6f56-47d7-abca-885eb642630c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.658149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8e706fd4-1e0c-461f-9428-51817581a3eb None None] MFA Rules not processed for user `8034f1ec47824eda9e4862fec718254f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:29.658489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce4486-6f56-47d7-abca-885eb642630c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.659391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8e706fd4-1e0c-461f-9428-51817581a3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:29.660184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce4486-6f56-47d7-abca-885eb642630c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.670992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1104/2202] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:29.672217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce4486-6f56-47d7-abca-885eb642630c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.683547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce4486-6f56-47d7-abca-885eb642630c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.689430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.690060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.690341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.702605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.702852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:29.703180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.707136 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.731897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1105/2203] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:29.732963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.745924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c18e1-1b39-4d1f-9737-441a37a6a8b8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.751902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ec56e-36bb-420e-883d-fc22a237e0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.754860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec56e-36bb-420e-883d-fc22a237e0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.754860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec56e-36bb-420e-883d-fc22a237e0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.754860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec56e-36bb-420e-883d-fc22a237e0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.754860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ec56e-36bb-420e-883d-fc22a237e0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.755851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ec56e-36bb-420e-883d-fc22a237e0e6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.767929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1106/2204] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:29.772076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ec56e-36bb-420e-883d-fc22a237e0e6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.786568 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ec56e-36bb-420e-883d-fc22a237e0e6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.794960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd4b7c-b619-4c6c-8cd4-a72cab7fcc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.795904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd4b7c-b619-4c6c-8cd4-a72cab7fcc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.796627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd4b7c-b619-4c6c-8cd4-a72cab7fcc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.797040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd4b7c-b619-4c6c-8cd4-a72cab7fcc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.798363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd4b7c-b619-4c6c-8cd4-a72cab7fcc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.801229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd4b7c-b619-4c6c-8cd4-a72cab7fcc40 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.818692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1107/2205] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:29.820299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd4b7c-b619-4c6c-8cd4-a72cab7fcc40 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.838747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1099/2206] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:29.840568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e706fd4-1e0c-461f-9428-51817581a3eb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.852636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e706fd4-1e0c-461f-9428-51817581a3eb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.855030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd4b7c-b619-4c6c-8cd4-a72cab7fcc40 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.858273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237fed-297d-4603-96f3-384636d168e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.858898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237fed-297d-4603-96f3-384636d168e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.859469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237fed-297d-4603-96f3-384636d168e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.859747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237fed-297d-4603-96f3-384636d168e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.860496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237fed-297d-4603-96f3-384636d168e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.862762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237fed-297d-4603-96f3-384636d168e5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.882270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.883163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.883751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.884319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.888222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1100/2207] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:29.888222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237fed-297d-4603-96f3-384636d168e5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.888480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.889799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.922158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237fed-297d-4603-96f3-384636d168e5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:29.935492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f73d5-f868-46fd-9740-af369abcbc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:29.936187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f73d5-f868-46fd-9740-af369abcbc18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:29.936473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f73d5-f868-46fd-9740-af369abcbc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:29.936751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f73d5-f868-46fd-9740-af369abcbc18 admin admin] PATH_INFO: `/v3/projects/ebd8c674a6d141fb83fa97702c986996/users/27c56c49e27c440181744bb67e178615/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:29.942727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1108/2208] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:29 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:29.945646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:29.949515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:29.973745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f73d5-f868-46fd-9740-af369abcbc18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd8c674a6d141fb83fa97702c986996, user_id=27c56c49e27c440181744bb67e178615, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:29.976388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f73d5-f868-46fd-9740-af369abcbc18 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:29.994162 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1101/2209] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:29 2022] PUT /identity/v3/projects/ebd8c674a6d141fb83fa97702c986996/users/27c56c49e27c440181744bb67e178615/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:30.006200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f73d5-f868-46fd-9740-af369abcbc18 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.013018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:30.042895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d1733-88a4-4f62-9001-31bd9c0a041a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.103263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f73d5-f868-46fd-9740-af369abcbc18 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.109346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6425-574a-495e-b1b8-fdef668f53a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.110057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6425-574a-495e-b1b8-fdef668f53a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.110510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6425-574a-495e-b1b8-fdef668f53a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.110783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6425-574a-495e-b1b8-fdef668f53a8 admin admin] PATH_INFO: `/v3/projects/530f7f7e1e7a42fdbaea8ea77f3ee70c/users/fae8adf8c2a742a3a603cb7483f81914/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.135312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c6425-574a-495e-b1b8-fdef668f53a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530f7f7e1e7a42fdbaea8ea77f3ee70c, user_id=fae8adf8c2a742a3a603cb7483f81914, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.138645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c6425-574a-495e-b1b8-fdef668f53a8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.140644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f9f7e-baf7-49c7-9900-5988ff87b82d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.141884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f9f7e-baf7-49c7-9900-5988ff87b82d service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.142473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f9f7e-baf7-49c7-9900-5988ff87b82d service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.143095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f9f7e-baf7-49c7-9900-5988ff87b82d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.185713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1102/2210] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:29 2022] PUT /identity/v3/projects/530f7f7e1e7a42fdbaea8ea77f3ee70c/users/fae8adf8c2a742a3a603cb7483f81914/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:30.189062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6425-574a-495e-b1b8-fdef668f53a8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.264505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f9f7e-baf7-49c7-9900-5988ff87b82d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.266187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f9f7e-baf7-49c7-9900-5988ff87b82d service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.271437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:30.294317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1109/2211] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:30.296990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682797-8df1-41af-9176-fb6f92f3375e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.298009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682797-8df1-41af-9176-fb6f92f3375e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.298686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682797-8df1-41af-9176-fb6f92f3375e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.334761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6425-574a-495e-b1b8-fdef668f53a8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.342313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312bf82-44e6-4111-b736-b7d9c669eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.343137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312bf82-44e6-4111-b736-b7d9c669eec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.343483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312bf82-44e6-4111-b736-b7d9c669eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.343761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312bf82-44e6-4111-b736-b7d9c669eec9 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.344099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c1682797-8df1-41af-9176-fb6f92f3375e None None] MFA Rules not processed for user `27c56c49e27c440181744bb67e178615`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:30.344507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312bf82-44e6-4111-b736-b7d9c669eec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.345776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c1682797-8df1-41af-9176-fb6f92f3375e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:30.369459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312bf82-44e6-4111-b736-b7d9c669eec9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.434087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1103/2212] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:30 2022] POST /identity/v3/users => generated 287 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:30.435245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cf470-63c3-48ef-952c-b736061298b7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.435541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cf470-63c3-48ef-952c-b736061298b7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.435818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cf470-63c3-48ef-952c-b736061298b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.488376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1110/2213] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:30.488376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1682797-8df1-41af-9176-fb6f92f3375e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.501509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1682797-8df1-41af-9176-fb6f92f3375e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.507567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd070af0-a94a-449a-a232-376ae46b39fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.508194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070af0-a94a-449a-a232-376ae46b39fe admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.508599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070af0-a94a-449a-a232-376ae46b39fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.508798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070af0-a94a-449a-a232-376ae46b39fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.509428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd070af0-a94a-449a-a232-376ae46b39fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.512097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd070af0-a94a-449a-a232-376ae46b39fe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.515341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9c6cf470-63c3-48ef-952c-b736061298b7 None None] MFA Rules not processed for user `fae8adf8c2a742a3a603cb7483f81914`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:30.516720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9c6cf470-63c3-48ef-952c-b736061298b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:30.522768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1111/2214] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:30.523823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd070af0-a94a-449a-a232-376ae46b39fe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.619307 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1104/2215] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:30.620121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cf470-63c3-48ef-952c-b736061298b7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.646024 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd070af0-a94a-449a-a232-376ae46b39fe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.646273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cf470-63c3-48ef-952c-b736061298b7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.654750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.655712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.656308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.656827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.658672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:30.660148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc8889-df10-4856-8d1e-6ea248991368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.660941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8889-df10-4856-8d1e-6ea248991368 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.661211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8889-df10-4856-8d1e-6ea248991368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.661484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8889-df10-4856-8d1e-6ea248991368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.662173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc8889-df10-4856-8d1e-6ea248991368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.664397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc8889-df10-4856-8d1e-6ea248991368 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.671000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:30.711590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1105/2216] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:30 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:30.712660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc8889-df10-4856-8d1e-6ea248991368 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.740733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.742976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.754472 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1112/2217] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:30.755539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.778753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82a1d6-7a10-475a-9f59-dc424ab603fa service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.812761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc8889-df10-4856-8d1e-6ea248991368 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.822386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8761e-c475-4f22-a2ab-f554884676e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.824257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8761e-c475-4f22-a2ab-f554884676e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.824569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8761e-c475-4f22-a2ab-f554884676e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.825438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8761e-c475-4f22-a2ab-f554884676e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.827729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8761e-c475-4f22-a2ab-f554884676e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.831537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8761e-c475-4f22-a2ab-f554884676e4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.837648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:30.854653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1106/2218] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:30.855990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8761e-c475-4f22-a2ab-f554884676e4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.878443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8761e-c475-4f22-a2ab-f554884676e4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:30.884180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18fb61-6c44-4a23-84a6-5067e741297f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.884381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a67d-83f6-42a4-9b49-7d8da21cd844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:30.884982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a67d-83f6-42a4-9b49-7d8da21cd844 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.885292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a67d-83f6-42a4-9b49-7d8da21cd844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.885614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a67d-83f6-42a4-9b49-7d8da21cd844 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.886109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18fb61-6c44-4a23-84a6-5067e741297f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:30.886439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8a67d-83f6-42a4-9b49-7d8da21cd844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.886654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18fb61-6c44-4a23-84a6-5067e741297f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:30.886873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18fb61-6c44-4a23-84a6-5067e741297f admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:30.887549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18fb61-6c44-4a23-84a6-5067e741297f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:30.888622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8a67d-83f6-42a4-9b49-7d8da21cd844 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.895575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18fb61-6c44-4a23-84a6-5067e741297f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:30.923078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1107/2219] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:30 2022] POST /identity/v3/users => generated 297 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:30.924059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18fb61-6c44-4a23-84a6-5067e741297f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.931412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1113/2220] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:30 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:30.931412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a67d-83f6-42a4-9b49-7d8da21cd844 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:30.937787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18fb61-6c44-4a23-84a6-5067e741297f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.064610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5d150-b498-4000-942d-af0f0311f0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.065313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5d150-b498-4000-942d-af0f0311f0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.065641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5d150-b498-4000-942d-af0f0311f0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.066012 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5d150-b498-4000-942d-af0f0311f0d8 admin admin] PATH_INFO: `/v3/projects/5fca452eedc6467ab85d03f7ff0e8765/users/b2af6c4f7f354067b27b887e4436be3e/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.109528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5d150-b498-4000-942d-af0f0311f0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fca452eedc6467ab85d03f7ff0e8765, user_id=b2af6c4f7f354067b27b887e4436be3e, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.116484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5d150-b498-4000-942d-af0f0311f0d8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.118593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a67d-83f6-42a4-9b49-7d8da21cd844 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.135427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c6918-c8a8-4d2d-8261-9920a17dfbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.139900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c6918-c8a8-4d2d-8261-9920a17dfbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.140368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c6918-c8a8-4d2d-8261-9920a17dfbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.140801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c6918-c8a8-4d2d-8261-9920a17dfbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.141922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c6918-c8a8-4d2d-8261-9920a17dfbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.144768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c6918-c8a8-4d2d-8261-9920a17dfbe6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.160632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1114/2222] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:31.160632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c6918-c8a8-4d2d-8261-9920a17dfbe6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.161034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1108/2222] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:30 2022] PUT /identity/v3/projects/5fca452eedc6467ab85d03f7ff0e8765/users/b2af6c4f7f354067b27b887e4436be3e/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:31.175074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ac9b1-3353-48e9-94f9-767c6750f693 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.175074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ac9b1-3353-48e9-94f9-767c6750f693 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.175074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ac9b1-3353-48e9-94f9-767c6750f693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.200524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c6918-c8a8-4d2d-8261-9920a17dfbe6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.233347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ab7ac9b1-3353-48e9-94f9-767c6750f693 None None] MFA Rules not processed for user `b2af6c4f7f354067b27b887e4436be3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:31.234686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ab7ac9b1-3353-48e9-94f9-767c6750f693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:31.326749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:31.340850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc4980-9b8b-4c0c-bcf2-9a8232831cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.341657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc4980-9b8b-4c0c-bcf2-9a8232831cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.342020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc4980-9b8b-4c0c-bcf2-9a8232831cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.342270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc4980-9b8b-4c0c-bcf2-9a8232831cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.343091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc4980-9b8b-4c0c-bcf2-9a8232831cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.345157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc4980-9b8b-4c0c-bcf2-9a8232831cd3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.383825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1109/2223] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:31.384737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1115/2224] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:31 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:31.388180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ac9b1-3353-48e9-94f9-767c6750f693 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.391192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc4980-9b8b-4c0c-bcf2-9a8232831cd3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.406663 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc4980-9b8b-4c0c-bcf2-9a8232831cd3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.413088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefe3dd-582b-415b-a7f4-2ebc09363198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.413965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefe3dd-582b-415b-a7f4-2ebc09363198 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.414433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefe3dd-582b-415b-a7f4-2ebc09363198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.414717 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ac9b1-3353-48e9-94f9-767c6750f693 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.414849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefe3dd-582b-415b-a7f4-2ebc09363198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.415578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefe3dd-582b-415b-a7f4-2ebc09363198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.417501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefe3dd-582b-415b-a7f4-2ebc09363198 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.424483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78b388-8ccb-4b2a-be30-ec406fd25035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.425167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78b388-8ccb-4b2a-be30-ec406fd25035 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.426071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78b388-8ccb-4b2a-be30-ec406fd25035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.426371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78b388-8ccb-4b2a-be30-ec406fd25035 admin admin] PATH_INFO: `/v3/projects/5783185a771f4c75bf75d64562d350b5/users/7f4b6e33700c4406a0e1a7531bdcf337/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.439417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1116/2225] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:31.440902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefe3dd-582b-415b-a7f4-2ebc09363198 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.466959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78b388-8ccb-4b2a-be30-ec406fd25035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5783185a771f4c75bf75d64562d350b5, user_id=7f4b6e33700c4406a0e1a7531bdcf337, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.469913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78b388-8ccb-4b2a-be30-ec406fd25035 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.508341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1110/2226] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:31 2022] PUT /identity/v3/projects/5783185a771f4c75bf75d64562d350b5/users/7f4b6e33700c4406a0e1a7531bdcf337/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:31.509160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78b388-8ccb-4b2a-be30-ec406fd25035 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.526743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:31.561274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefe3dd-582b-415b-a7f4-2ebc09363198 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.601573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78b388-8ccb-4b2a-be30-ec406fd25035 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.610566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10171b4-8d3b-4b88-9ca2-22d8796a169e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.611823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10171b4-8d3b-4b88-9ca2-22d8796a169e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.612354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10171b4-8d3b-4b88-9ca2-22d8796a169e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.616527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10171b4-8d3b-4b88-9ca2-22d8796a169e admin admin] PATH_INFO: `/v3/projects/abc088805cc84f62978b1b0a102f1bc2/users/22ba02bfbf96428fbd8b8a55531c0a87/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.633044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea38b1-32e4-48fa-86df-bf0f0721b930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.633971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea38b1-32e4-48fa-86df-bf0f0721b930 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.634214 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea38b1-32e4-48fa-86df-bf0f0721b930 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.634658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea38b1-32e4-48fa-86df-bf0f0721b930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.652215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10171b4-8d3b-4b88-9ca2-22d8796a169e admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc088805cc84f62978b1b0a102f1bc2, user_id=22ba02bfbf96428fbd8b8a55531c0a87, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.654869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10171b4-8d3b-4b88-9ca2-22d8796a169e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.678003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1111/2227] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:31 2022] PUT /identity/v3/projects/abc088805cc84f62978b1b0a102f1bc2/users/22ba02bfbf96428fbd8b8a55531c0a87/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:31.679047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10171b4-8d3b-4b88-9ca2-22d8796a169e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.735167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea38b1-32e4-48fa-86df-bf0f0721b930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.736899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea38b1-32e4-48fa-86df-bf0f0721b930 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.749984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1117/2228] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:31.751121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea38b1-32e4-48fa-86df-bf0f0721b930 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.767500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea38b1-32e4-48fa-86df-bf0f0721b930 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.768584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10171b4-8d3b-4b88-9ca2-22d8796a169e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.774077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc345a3-d374-4f35-883c-ab5de1b2b1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.774522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b815f0-76f2-4756-ab12-4560151c7ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.774696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc345a3-d374-4f35-883c-ab5de1b2b1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.774975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc345a3-d374-4f35-883c-ab5de1b2b1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.775149 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b815f0-76f2-4756-ab12-4560151c7ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.775291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc345a3-d374-4f35-883c-ab5de1b2b1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.775464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b815f0-76f2-4756-ab12-4560151c7ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.775748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b815f0-76f2-4756-ab12-4560151c7ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.776160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc345a3-d374-4f35-883c-ab5de1b2b1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.776643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b815f0-76f2-4756-ab12-4560151c7ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.778467 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b815f0-76f2-4756-ab12-4560151c7ac6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.779413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc345a3-d374-4f35-883c-ab5de1b2b1e9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.791283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1112/2229] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:31.799517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b815f0-76f2-4756-ab12-4560151c7ac6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.805872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1118/2230] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:31.806981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc345a3-d374-4f35-883c-ab5de1b2b1e9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.816978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b815f0-76f2-4756-ab12-4560151c7ac6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.822263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.822510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc345a3-d374-4f35-883c-ab5de1b2b1e9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.823042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.823443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.823767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] PATH_INFO: `/v3/users/b2af6c4f7f354067b27b887e4436be3e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.825337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2af6c4f7f354067b27b887e4436be3e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.827900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.832860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f26870-1f22-431e-b0b4-1bf766fe3b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.833666 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f26870-1f22-431e-b0b4-1bf766fe3b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.834038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f26870-1f22-431e-b0b4-1bf766fe3b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.834381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f26870-1f22-431e-b0b4-1bf766fe3b01 admin admin] PATH_INFO: `/v3/projects/5783185a771f4c75bf75d64562d350b5/users/7f4b6e33700c4406a0e1a7531bdcf337/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.836781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f26870-1f22-431e-b0b4-1bf766fe3b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5783185a771f4c75bf75d64562d350b5, user_id=7f4b6e33700c4406a0e1a7531bdcf337, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:31.841245 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f26870-1f22-431e-b0b4-1bf766fe3b01 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:31.860218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1119/2231] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:31 2022] PUT /identity/v3/projects/5783185a771f4c75bf75d64562d350b5/users/7f4b6e33700c4406a0e1a7531bdcf337/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:31.861197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f26870-1f22-431e-b0b4-1bf766fe3b01 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:31.889853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:31.917984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2af6c4f7f354067b27b887e4436be3e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:31.919185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2af6c4f7f354067b27b887e4436be3e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:31.934566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2af6c4f7f354067b27b887e4436be3e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:31.942843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2af6c4f7f354067b27b887e4436be3e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:31.979316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f26870-1f22-431e-b0b4-1bf766fe3b01 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:31.981402 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ade20e-7ea7-4140-81b9-1305d4bfd0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2af6c4f7f354067b27b887e4436be3e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:31.991740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1113/2232] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:31 2022] DELETE /identity/v3/users/b2af6c4f7f354067b27b887e4436be3e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:31.993237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7b5fb-021d-442e-82c2-3d72a4a3d6a7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.993552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7b5fb-021d-442e-82c2-3d72a4a3d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:31.993794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7b5fb-021d-442e-82c2-3d72a4a3d6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:31.998596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27fb3-4265-451d-869a-fb4436193368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:31.999533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27fb3-4265-451d-869a-fb4436193368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:31.999662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27fb3-4265-451d-869a-fb4436193368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:32.000314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27fb3-4265-451d-869a-fb4436193368 admin admin] PATH_INFO: `/v3/projects/abc088805cc84f62978b1b0a102f1bc2/users/22ba02bfbf96428fbd8b8a55531c0a87/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:32.003761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27fb3-4265-451d-869a-fb4436193368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc088805cc84f62978b1b0a102f1bc2, user_id=22ba02bfbf96428fbd8b8a55531c0a87, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:32.011650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27fb3-4265-451d-869a-fb4436193368 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:32.029771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1120/2233] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:31 2022] PUT /identity/v3/projects/abc088805cc84f62978b1b0a102f1bc2/users/22ba02bfbf96428fbd8b8a55531c0a87/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:32.031139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f646e3-683f-4749-a7dc-31a507477d69 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:32.031584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f646e3-683f-4749-a7dc-31a507477d69 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:32.031858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f646e3-683f-4749-a7dc-31a507477d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:32.040515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-aac7b5fb-021d-442e-82c2-3d72a4a3d6a7 None None] MFA Rules not processed for user `7f4b6e33700c4406a0e1a7531bdcf337`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:32.041827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aac7b5fb-021d-442e-82c2-3d72a4a3d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.064392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a9f646e3-683f-4749-a7dc-31a507477d69 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:32.065898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a9f646e3-683f-4749-a7dc-31a507477d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.075281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:32.136696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1114/2234] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:32.137347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151d99-0fea-4744-9e5d-ead395ab4dac None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:32.137904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151d99-0fea-4744-9e5d-ead395ab4dac None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:32.138946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151d99-0fea-4744-9e5d-ead395ab4dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:32.163559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1121/2235] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:32.164444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f646e3-683f-4749-a7dc-31a507477d69 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:32.165773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a9f646e3-683f-4749-a7dc-31a507477d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.181246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-59151d99-0fea-4744-9e5d-ead395ab4dac None None] MFA Rules not processed for user `22ba02bfbf96428fbd8b8a55531c0a87`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:32.182483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-59151d99-0fea-4744-9e5d-ead395ab4dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.243974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f646e3-683f-4749-a7dc-31a507477d69 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:32.248497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:32.255613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:32.256093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:32.256466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:32.274085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:32.275977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:32.277588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1115/2236] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:32.284219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151d99-0fea-4744-9e5d-ead395ab4dac None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:32.286258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-59151d99-0fea-4744-9e5d-ead395ab4dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.295370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1122/2237] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:32.297971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:32.297971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.310657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:32.322000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39be02-4579-4fd3-a179-371a7db2929a service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:32.327157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:32.327814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:32.328425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:32.328712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:32.330843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.361690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151d99-0fea-4744-9e5d-ead395ab4dac None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:32.373146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69babad4-04e7-4bc2-a12a-6e6b34cce4bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:32.374003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69babad4-04e7-4bc2-a12a-6e6b34cce4bf service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:32.374351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69babad4-04e7-4bc2-a12a-6e6b34cce4bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:32.374690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69babad4-04e7-4bc2-a12a-6e6b34cce4bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:32.386078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69babad4-04e7-4bc2-a12a-6e6b34cce4bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:32.387577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69babad4-04e7-4bc2-a12a-6e6b34cce4bf service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:32.416664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1116/2238] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:32.424829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:32.426282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:32.437132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1123/2239] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:32.787575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69babad4-04e7-4bc2-a12a-6e6b34cce4bf service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:32.799378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69babad4-04e7-4bc2-a12a-6e6b34cce4bf service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:32.803141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a3a5f-eefc-4f8e-9f94-425f63f5e741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:32.803816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a3a5f-eefc-4f8e-9f94-425f63f5e741 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:32.804159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a3a5f-eefc-4f8e-9f94-425f63f5e741 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:32.804512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a3a5f-eefc-4f8e-9f94-425f63f5e741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:32.807096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-827a3a5f-eefc-4f8e-9f94-425f63f5e741 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.895693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:32.897606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:32.919481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827de00e-cf10-4fff-9152-d64b6e2f93c7 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:32.934554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:32.936501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:32.938839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:32.940301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] PATH_INFO: `/v3/projects/5fca452eedc6467ab85d03f7ff0e8765` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:32.945243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fca452eedc6467ab85d03f7ff0e8765)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:32.947729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:32.949933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a3a5f-eefc-4f8e-9f94-425f63f5e741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:32.951650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a3a5f-eefc-4f8e-9f94-425f63f5e741 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:32.963950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1117/2240] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:33.025296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:33.027295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fca452eedc6467ab85d03f7ff0e8765', 'request_id': 'req-384eaaba-1033-43d2-a9f9-8b23679b7e66'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.037407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fca452eedc6467ab85d03f7ff0e8765', 'request_id': 'req-384eaaba-1033-43d2-a9f9-8b23679b7e66'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.047142 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fca452eedc6467ab85d03f7ff0e8765', 'request_id': 'req-384eaaba-1033-43d2-a9f9-8b23679b7e66'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.063433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] The token cache is being invalidate because project 5fca452eedc6467ab85d03f7ff0e8765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:33.063433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.063433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-384eaaba-1033-43d2-a9f9-8b23679b7e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.065179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1124/2241] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:32 2022] DELETE /identity/v3/projects/5fca452eedc6467ab85d03f7ff0e8765 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:33.070093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ec56-3f1e-41ec-a1f8-60d5818f2b80 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.070451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ec56-3f1e-41ec-a1f8-60d5818f2b80 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.070794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ec56-3f1e-41ec-a1f8-60d5818f2b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.093354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c268ec56-3f1e-41ec-a1f8-60d5818f2b80 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:33.096591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c268ec56-3f1e-41ec-a1f8-60d5818f2b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:33.198881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1118/2242] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:33.201149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d089c25-9325-45eb-90e2-31e3d6302be3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.201706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d089c25-9325-45eb-90e2-31e3d6302be3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.202555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d089c25-9325-45eb-90e2-31e3d6302be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.250755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8d089c25-9325-45eb-90e2-31e3d6302be3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:33.252161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8d089c25-9325-45eb-90e2-31e3d6302be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:33.269422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1125/2243] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:33.272333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268ec56-3f1e-41ec-a1f8-60d5818f2b80 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.285547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268ec56-3f1e-41ec-a1f8-60d5818f2b80 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:33.291357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:33.292313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.292882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.293427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.294289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:33.294762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:33.297210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:33.312385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1119/2244] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:33.316868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d089c25-9325-45eb-90e2-31e3d6302be3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.328318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d089c25-9325-45eb-90e2-31e3d6302be3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:33.341131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1a4d0-c2e6-48ff-9a8c-bb3549015000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:33.341131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a4d0-c2e6-48ff-9a8c-bb3549015000 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.341131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a4d0-c2e6-48ff-9a8c-bb3549015000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.341364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a4d0-c2e6-48ff-9a8c-bb3549015000 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.342241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1a4d0-c2e6-48ff-9a8c-bb3549015000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:33.345465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1a4d0-c2e6-48ff-9a8c-bb3549015000 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:33.360411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.362658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:33.378592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dca6c-71e1-4152-90ad-1b985f20b6cd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:33.383748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:33.384483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.384849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.385204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] PATH_INFO: `/v3/users/27c56c49e27c440181744bb67e178615` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.386530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c56c49e27c440181744bb67e178615)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:33.388623 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:33.393047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1126/2245] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:33 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:33.397170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1a4d0-c2e6-48ff-9a8c-bb3549015000 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.450447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:33.464163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c56c49e27c440181744bb67e178615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.464783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c56c49e27c440181744bb67e178615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.478407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c56c49e27c440181744bb67e178615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.489361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c56c49e27c440181744bb67e178615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.500516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a04ccc72-117b-4115-8aff-dadc72e06834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c56c49e27c440181744bb67e178615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:33.508806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1a4d0-c2e6-48ff-9a8c-bb3549015000 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:33.511305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1120/2246] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:33 2022] DELETE /identity/v3/users/27c56c49e27c440181744bb67e178615 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:33.513960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f48f7d-fd3d-45cf-af3f-8bd1427b94a0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.515704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f48f7d-fd3d-45cf-af3f-8bd1427b94a0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.516080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f48f7d-fd3d-45cf-af3f-8bd1427b94a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.536925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e3f48f7d-fd3d-45cf-af3f-8bd1427b94a0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:33.538166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e3f48f7d-fd3d-45cf-af3f-8bd1427b94a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:33.585534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:33.586160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.586436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.586708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.587413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:33.589561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:33.611046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1121/2247] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:33.612324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1127/2248] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:33 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:33.616085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f48f7d-fd3d-45cf-af3f-8bd1427b94a0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.617469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e3f48f7d-fd3d-45cf-af3f-8bd1427b94a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:33.624907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.628275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:33.631169 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f48f7d-fd3d-45cf-af3f-8bd1427b94a0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:33.637474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8703a-0e3d-4f57-bc8e-fcbb870b0874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:33.637474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8703a-0e3d-4f57-bc8e-fcbb870b0874 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.637474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8703a-0e3d-4f57-bc8e-fcbb870b0874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.637474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8703a-0e3d-4f57-bc8e-fcbb870b0874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.638186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8703a-0e3d-4f57-bc8e-fcbb870b0874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:33.639899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8703a-0e3d-4f57-bc8e-fcbb870b0874 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:33.652187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1122/2249] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:33.656274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8703a-0e3d-4f57-bc8e-fcbb870b0874 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.667406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8703a-0e3d-4f57-bc8e-fcbb870b0874 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:33.672742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eddb0-c47b-47af-aa8c-fe950e4893a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:33.673382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eddb0-c47b-47af-aa8c-fe950e4893a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.673660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eddb0-c47b-47af-aa8c-fe950e4893a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.673927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eddb0-c47b-47af-aa8c-fe950e4893a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.674574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7eddb0-c47b-47af-aa8c-fe950e4893a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:33.676220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7eddb0-c47b-47af-aa8c-fe950e4893a2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:33.690027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1123/2250] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:20:33.691124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eddb0-c47b-47af-aa8c-fe950e4893a2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.703257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eddb0-c47b-47af-aa8c-fe950e4893a2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:33.706989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:33.710069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785110fa-71a5-4606-9464-f59a3cf2a0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:33.710756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785110fa-71a5-4606-9464-f59a3cf2a0a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.711056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785110fa-71a5-4606-9464-f59a3cf2a0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.711323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785110fa-71a5-4606-9464-f59a3cf2a0a3 admin admin] PATH_INFO: `/v3/projects/f2ffdb40e3f846e99cd245544b9aeebd/users/a560c330d47d440aa18214c330b1a7cb/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.730190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd44d1e-06a2-4325-b13f-0bce0c7e3359 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:33.730432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785110fa-71a5-4606-9464-f59a3cf2a0a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ffdb40e3f846e99cd245544b9aeebd, user_id=a560c330d47d440aa18214c330b1a7cb, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:33.733086 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785110fa-71a5-4606-9464-f59a3cf2a0a3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:33.737230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:33.738135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.739615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.739905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.748393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1124/2251] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:33 2022] PUT /identity/v3/projects/f2ffdb40e3f846e99cd245544b9aeebd/users/a560c330d47d440aa18214c330b1a7cb/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:33.752041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:33.753686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:33.759965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4e92-9279-4cd2-8651-e6780c514d3c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:33.760299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4e92-9279-4cd2-8651-e6780c514d3c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:33.760578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4e92-9279-4cd2-8651-e6780c514d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:33.766664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1128/2252] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:33.808615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-48bb4e92-9279-4cd2-8651-e6780c514d3c None None] MFA Rules not processed for user `a560c330d47d440aa18214c330b1a7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:33.810735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-48bb4e92-9279-4cd2-8651-e6780c514d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:33.911594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:33.924213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1125/2253] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:33.936753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:33.938334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:34.044349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed91f5-f226-4570-a14d-62793eb95169 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:34.160242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd90a-0407-48a1-acc9-9b9508450ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:34.160954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd90a-0407-48a1-acc9-9b9508450ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:34.161318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd90a-0407-48a1-acc9-9b9508450ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:34.161652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd90a-0407-48a1-acc9-9b9508450ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:34.177910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd90a-0407-48a1-acc9-9b9508450ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:34.180152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd90a-0407-48a1-acc9-9b9508450ec3 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:34.213220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1129/2254] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:34.429728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb4e92-9279-4cd2-8651-e6780c514d3c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:34.431451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-48bb4e92-9279-4cd2-8651-e6780c514d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:34.449308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd90a-0407-48a1-acc9-9b9508450ec3 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:34.462474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd90a-0407-48a1-acc9-9b9508450ec3 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:34.476899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:34.477536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:34.477822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:34.478098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:34.480812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:34.527305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb4e92-9279-4cd2-8651-e6780c514d3c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:34.532167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:34.532889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:34.533092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:34.533784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] PATH_INFO: `/v3/projects/ebd8c674a6d141fb83fa97702c986996` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:34.534522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd8c674a6d141fb83fa97702c986996)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:34.536506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:34.615327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:34.618602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:34.627147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd8c674a6d141fb83fa97702c986996', 'request_id': 'req-77b5b0af-8a51-416b-ac85-8b5acc377bff'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.634710 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1130/2255] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:34.637558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd8c674a6d141fb83fa97702c986996', 'request_id': 'req-77b5b0af-8a51-416b-ac85-8b5acc377bff'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.646167 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd8c674a6d141fb83fa97702c986996', 'request_id': 'req-77b5b0af-8a51-416b-ac85-8b5acc377bff'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.671550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] The token cache is being invalidate because project ebd8c674a6d141fb83fa97702c986996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:34.672196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.673225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.675663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1126/2256] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:34 2022] DELETE /identity/v3/projects/ebd8c674a6d141fb83fa97702c986996 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:34.680423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:34.698153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea2f9d-0dc0-4d18-9f8e-3e4045bd20a3 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:34.752805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:34.772804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:34.774183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:34.775091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:34.775418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] PATH_INFO: `/v3/users/7f4b6e33700c4406a0e1a7531bdcf337` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:34.777299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4b6e33700c4406a0e1a7531bdcf337)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:34.780763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:34.861733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4b6e33700c4406a0e1a7531bdcf337'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.862921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4b6e33700c4406a0e1a7531bdcf337'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.881893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4b6e33700c4406a0e1a7531bdcf337'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.883856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4b6e33700c4406a0e1a7531bdcf337'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.901505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4b6e33700c4406a0e1a7531bdcf337'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:34.929579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1131/2257] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:34 2022] DELETE /identity/v3/users/7f4b6e33700c4406a0e1a7531bdcf337 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:35.367503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:35.369969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:35.396964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5b0af-8a51-416b-ac85-8b5acc377bff admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:35.520135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:35.521131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:35.521580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:35.522020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] PATH_INFO: `/v3/projects/5783185a771f4c75bf75d64562d350b5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:35.524013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5783185a771f4c75bf75d64562d350b5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:35.527291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:35.550120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:35.552535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:35.602046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:35.624703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5783185a771f4c75bf75d64562d350b5', 'request_id': 'req-75d3eb51-d58b-4345-a543-9457efae4dd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:35.633759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5783185a771f4c75bf75d64562d350b5', 'request_id': 'req-75d3eb51-d58b-4345-a543-9457efae4dd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:35.643507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5783185a771f4c75bf75d64562d350b5', 'request_id': 'req-75d3eb51-d58b-4345-a543-9457efae4dd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:35.662299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068de8-613a-40c1-9d81-b1763574804b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:35.667815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] The token cache is being invalidate because project 5783185a771f4c75bf75d64562d350b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:35.668389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:35.669491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-75d3eb51-d58b-4345-a543-9457efae4dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:35.672800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1127/2258] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:35 2022] DELETE /identity/v3/projects/5783185a771f4c75bf75d64562d350b5 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:35.678726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb4968-8a70-4c16-b766-6d38bb1ae325 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:35.679008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb4968-8a70-4c16-b766-6d38bb1ae325 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:35.679302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb4968-8a70-4c16-b766-6d38bb1ae325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:35.742005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-adfb4968-8a70-4c16-b766-6d38bb1ae325 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:35.744949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:35.745669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:35.745951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:35.746224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:35.748411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-adfb4968-8a70-4c16-b766-6d38bb1ae325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:35.750903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:35.815109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:35.855395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1128/2259] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:35.857266 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dba6b0-ee34-4fa3-bc88-6db9632ad58f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:35.857619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dba6b0-ee34-4fa3-bc88-6db9632ad58f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:35.857958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dba6b0-ee34-4fa3-bc88-6db9632ad58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:35.897190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:35.899356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:35.908540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e1dba6b0-ee34-4fa3-bc88-6db9632ad58f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:35.910421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e1dba6b0-ee34-4fa3-bc88-6db9632ad58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:35.916328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1132/2260] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:35.920592 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1129/2261] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:35.925856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:35.943186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1f262-9137-4034-9517-d5af791729de service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:35.949004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:35.949680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:35.950020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:35.950303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:35.951091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:35.951583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:35.953408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.004640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1133/2262] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:36.005797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dba6b0-ee34-4fa3-bc88-6db9632ad58f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.018479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dba6b0-ee34-4fa3-bc88-6db9632ad58f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.024112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dc287-7295-49ad-8734-2cdabdefaa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.024662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dc287-7295-49ad-8734-2cdabdefaa77 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.024938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dc287-7295-49ad-8734-2cdabdefaa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.025209 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dc287-7295-49ad-8734-2cdabdefaa77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.025926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dc287-7295-49ad-8734-2cdabdefaa77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.030000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dc287-7295-49ad-8734-2cdabdefaa77 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.073943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1130/2263] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:36 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:20:36.078757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.100913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127a6c0-1a34-4f7f-8a64-a3f127da7cad admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.235431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9eafc-8c4c-4442-b42a-15130fda13f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.236537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9eafc-8c4c-4442-b42a-15130fda13f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.237025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9eafc-8c4c-4442-b42a-15130fda13f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.237497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9eafc-8c4c-4442-b42a-15130fda13f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.238532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9eafc-8c4c-4442-b42a-15130fda13f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.240592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9eafc-8c4c-4442-b42a-15130fda13f1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.268864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1134/2264] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:36 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:36.274658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dc287-7295-49ad-8734-2cdabdefaa77 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.287097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dc287-7295-49ad-8734-2cdabdefaa77 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.299351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3a463-3e3c-4916-afe9-49e2f89dbeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.302622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3a463-3e3c-4916-afe9-49e2f89dbeac admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.305722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3a463-3e3c-4916-afe9-49e2f89dbeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.306415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3a463-3e3c-4916-afe9-49e2f89dbeac admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.307560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3a463-3e3c-4916-afe9-49e2f89dbeac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.311112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3a463-3e3c-4916-afe9-49e2f89dbeac admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.334519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1131/2265] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:36.339618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9eafc-8c4c-4442-b42a-15130fda13f1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.357119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9eafc-8c4c-4442-b42a-15130fda13f1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.369729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b07f2-3179-4bb3-988d-5cde56634e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.370786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b07f2-3179-4bb3-988d-5cde56634e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.371362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b07f2-3179-4bb3-988d-5cde56634e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.371506 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b07f2-3179-4bb3-988d-5cde56634e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.372156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b07f2-3179-4bb3-988d-5cde56634e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.373885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b07f2-3179-4bb3-988d-5cde56634e14 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.389794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1135/2266] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:36.395030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3a463-3e3c-4916-afe9-49e2f89dbeac admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.417699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3a463-3e3c-4916-afe9-49e2f89dbeac admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.431784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc4e74-2652-4482-83fc-990db54bd296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.433760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc4e74-2652-4482-83fc-990db54bd296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.434732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc4e74-2652-4482-83fc-990db54bd296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.435486 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc4e74-2652-4482-83fc-990db54bd296 admin admin] PATH_INFO: `/v3/projects/94736040b12f4702bdc8426f7ec6d4eb/users/be5436999ce84450ac2557fc7110e0a1/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.457181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc4e74-2652-4482-83fc-990db54bd296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94736040b12f4702bdc8426f7ec6d4eb, user_id=be5436999ce84450ac2557fc7110e0a1, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.460464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc4e74-2652-4482-83fc-990db54bd296 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.486626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1132/2267] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:36 2022] PUT /identity/v3/projects/94736040b12f4702bdc8426f7ec6d4eb/users/be5436999ce84450ac2557fc7110e0a1/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:36.499257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839a8bc-fb5a-4498-985c-2fada4814990 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.499556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839a8bc-fb5a-4498-985c-2fada4814990 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.499860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839a8bc-fb5a-4498-985c-2fada4814990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.531652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5839a8bc-fb5a-4498-985c-2fada4814990 None None] MFA Rules not processed for user `be5436999ce84450ac2557fc7110e0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:36.532898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5839a8bc-fb5a-4498-985c-2fada4814990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:36.635121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1136/2268] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:36.640601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc4e74-2652-4482-83fc-990db54bd296 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.715272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:36.722748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc4e74-2652-4482-83fc-990db54bd296 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.730709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c5216-c5db-42fe-984b-8b0b5d040d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.731370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c5216-c5db-42fe-984b-8b0b5d040d2a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.731645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c5216-c5db-42fe-984b-8b0b5d040d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.731911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c5216-c5db-42fe-984b-8b0b5d040d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.732567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c5216-c5db-42fe-984b-8b0b5d040d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.734481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c5216-c5db-42fe-984b-8b0b5d040d2a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.765113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1133/2269] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:36 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:36.769223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839a8bc-fb5a-4498-985c-2fada4814990 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.781049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839a8bc-fb5a-4498-985c-2fada4814990 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.803801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:36.843313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3798-91da-403b-8ebd-cc043532f144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.843993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3798-91da-403b-8ebd-cc043532f144 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.844274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3798-91da-403b-8ebd-cc043532f144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.844550 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3798-91da-403b-8ebd-cc043532f144 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.845319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c3798-91da-403b-8ebd-cc043532f144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.847205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c3798-91da-403b-8ebd-cc043532f144 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.872246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1137/2270] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:36 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:36.876377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c5216-c5db-42fe-984b-8b0b5d040d2a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.889359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c5216-c5db-42fe-984b-8b0b5d040d2a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.895200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c56d22-f159-443c-a058-4136d5cf4af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.895938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c56d22-f159-443c-a058-4136d5cf4af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.896291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c56d22-f159-443c-a058-4136d5cf4af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.896951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c56d22-f159-443c-a058-4136d5cf4af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.897196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c56d22-f159-443c-a058-4136d5cf4af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.899336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c56d22-f159-443c-a058-4136d5cf4af1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.912210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1134/2271] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:36.917503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3798-91da-403b-8ebd-cc043532f144 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.928715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3798-91da-403b-8ebd-cc043532f144 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.933851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3467cbf-1d84-4758-a4fc-6c0c94e69aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.934469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467cbf-1d84-4758-a4fc-6c0c94e69aff admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.934781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467cbf-1d84-4758-a4fc-6c0c94e69aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.935052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467cbf-1d84-4758-a4fc-6c0c94e69aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.935741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3467cbf-1d84-4758-a4fc-6c0c94e69aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.937344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3467cbf-1d84-4758-a4fc-6c0c94e69aff admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:36.949665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1138/2272] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:36.954299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c56d22-f159-443c-a058-4136d5cf4af1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:36.967043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c56d22-f159-443c-a058-4136d5cf4af1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:36.972389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6e79f-28bf-4990-a83d-dc4f6b2b3b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:36.973144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6e79f-28bf-4990-a83d-dc4f6b2b3b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:36.973501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6e79f-28bf-4990-a83d-dc4f6b2b3b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:36.973837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6e79f-28bf-4990-a83d-dc4f6b2b3b6a admin admin] PATH_INFO: `/v3/projects/b802dd46cc0440bc8426bc0caeb3b943/users/de86781742b64c518f8b5e764fc0028b/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:36.993058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6e79f-28bf-4990-a83d-dc4f6b2b3b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b802dd46cc0440bc8426bc0caeb3b943, user_id=de86781742b64c518f8b5e764fc0028b, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:36.996238 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6e79f-28bf-4990-a83d-dc4f6b2b3b6a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:37.011866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1135/2273] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:36 2022] PUT /identity/v3/projects/b802dd46cc0440bc8426bc0caeb3b943/users/de86781742b64c518f8b5e764fc0028b/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:37.024022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ec1d3-b6bb-40df-96a7-17092cfa52c8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.024282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ec1d3-b6bb-40df-96a7-17092cfa52c8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.024541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ec1d3-b6bb-40df-96a7-17092cfa52c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.056894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2c4ec1d3-b6bb-40df-96a7-17092cfa52c8 None None] MFA Rules not processed for user `de86781742b64c518f8b5e764fc0028b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:37.058255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2c4ec1d3-b6bb-40df-96a7-17092cfa52c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:37.136797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1139/2274] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:37.141445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6e79f-28bf-4990-a83d-dc4f6b2b3b6a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:37.217488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6e79f-28bf-4990-a83d-dc4f6b2b3b6a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:37.223589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e6fb0-6c7e-4f6a-a3ee-6a9db2456b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:37.224673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e6fb0-6c7e-4f6a-a3ee-6a9db2456b6b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.224673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e6fb0-6c7e-4f6a-a3ee-6a9db2456b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.224854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e6fb0-6c7e-4f6a-a3ee-6a9db2456b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.225364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e6fb0-6c7e-4f6a-a3ee-6a9db2456b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:37.227335 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e6fb0-6c7e-4f6a-a3ee-6a9db2456b6b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:37.265614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1136/2275] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:37 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:37.269753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ec1d3-b6bb-40df-96a7-17092cfa52c8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:37.283835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ec1d3-b6bb-40df-96a7-17092cfa52c8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:37.356866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07cb82-d06d-43fd-9157-576679eab1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:37.357571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07cb82-d06d-43fd-9157-576679eab1bd admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.358052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07cb82-d06d-43fd-9157-576679eab1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.358209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07cb82-d06d-43fd-9157-576679eab1bd admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.358749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07cb82-d06d-43fd-9157-576679eab1bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:37.360937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07cb82-d06d-43fd-9157-576679eab1bd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:37.384013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1140/2276] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:37 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:37.389941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e6fb0-6c7e-4f6a-a3ee-6a9db2456b6b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:37.401216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e6fb0-6c7e-4f6a-a3ee-6a9db2456b6b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:37.406777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b52ef-58f4-4a19-9877-f9ee10d6c6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:37.407426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b52ef-58f4-4a19-9877-f9ee10d6c6fb admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.407703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b52ef-58f4-4a19-9877-f9ee10d6c6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.407966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b52ef-58f4-4a19-9877-f9ee10d6c6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.408602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b52ef-58f4-4a19-9877-f9ee10d6c6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:37.410394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b52ef-58f4-4a19-9877-f9ee10d6c6fb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:37.427286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1137/2277] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:37.430614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07cb82-d06d-43fd-9157-576679eab1bd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:37.441367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07cb82-d06d-43fd-9157-576679eab1bd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:37.447752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc37ac1-bba7-4fcb-9e46-9a7d8d95fd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:37.448424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc37ac1-bba7-4fcb-9e46-9a7d8d95fd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.448689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc37ac1-bba7-4fcb-9e46-9a7d8d95fd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.448959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc37ac1-bba7-4fcb-9e46-9a7d8d95fd92 admin admin] PATH_INFO: `/v3/projects/e40ff3dac18c4abf8b9d7671c3757cff/users/379fc12ec534455fb76aa03d0b552df7/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.466021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc37ac1-bba7-4fcb-9e46-9a7d8d95fd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40ff3dac18c4abf8b9d7671c3757cff, user_id=379fc12ec534455fb76aa03d0b552df7, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:37.468817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc37ac1-bba7-4fcb-9e46-9a7d8d95fd92 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:37.480389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:37.500681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1141/2278] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:37 2022] PUT /identity/v3/projects/e40ff3dac18c4abf8b9d7671c3757cff/users/379fc12ec534455fb76aa03d0b552df7/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:37.505084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b52ef-58f4-4a19-9877-f9ee10d6c6fb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:37.536046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b52ef-58f4-4a19-9877-f9ee10d6c6fb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:37.626698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70d0c-5a0d-489c-9935-ca9a232fcf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:37.627350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70d0c-5a0d-489c-9935-ca9a232fcf56 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.628943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70d0c-5a0d-489c-9935-ca9a232fcf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.629403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70d0c-5a0d-489c-9935-ca9a232fcf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.630830 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff70d0c-5a0d-489c-9935-ca9a232fcf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:37.634436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff70d0c-5a0d-489c-9935-ca9a232fcf56 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:37.652678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1138/2279] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:37.661045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc37ac1-bba7-4fcb-9e46-9a7d8d95fd92 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:37.676507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc37ac1-bba7-4fcb-9e46-9a7d8d95fd92 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:37.691758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:37.692448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.692733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.693012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] PATH_INFO: `/v3/projects/e40ff3dac18c4abf8b9d7671c3757cff/users/379fc12ec534455fb76aa03d0b552df7/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.700662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40ff3dac18c4abf8b9d7671c3757cff, user_id=379fc12ec534455fb76aa03d0b552df7, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:37.706223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:37.735338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1142/2280] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:37 2022] PUT /identity/v3/projects/e40ff3dac18c4abf8b9d7671c3757cff/users/379fc12ec534455fb76aa03d0b552df7/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:37.746338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d184-8e27-4b34-bd2c-a8a167455567 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.746628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d184-8e27-4b34-bd2c-a8a167455567 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.746928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d184-8e27-4b34-bd2c-a8a167455567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.756610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:37.758901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:37.787543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6a15d184-8e27-4b34-bd2c-a8a167455567 None None] MFA Rules not processed for user `379fc12ec534455fb76aa03d0b552df7`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:37.789018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6a15d184-8e27-4b34-bd2c-a8a167455567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:37.794131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:37.849861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737188c-9344-460b-b001-11a09d0dfae0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:37.855354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:37.855892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.856317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.856445 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] PATH_INFO: `/v3/users/f8228d72e0d044ed9a78574bbc5d718b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.857607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8228d72e0d044ed9a78574bbc5d718b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:37.859571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:37.894273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1139/2281] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:37.903806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15d184-8e27-4b34-bd2c-a8a167455567 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:37.905861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6a15d184-8e27-4b34-bd2c-a8a167455567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:37.921860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8228d72e0d044ed9a78574bbc5d718b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:37.922610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8228d72e0d044ed9a78574bbc5d718b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:37.933370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8228d72e0d044ed9a78574bbc5d718b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:37.934679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8228d72e0d044ed9a78574bbc5d718b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:37.952006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1703b1f9-a49b-4d0b-ba2f-eebbbb104913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8228d72e0d044ed9a78574bbc5d718b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:37.967644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1143/2282] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:37 2022] DELETE /identity/v3/users/f8228d72e0d044ed9a78574bbc5d718b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:37.973332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520782d5-79b8-4c89-b16e-61e23056ab62 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:37.973807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520782d5-79b8-4c89-b16e-61e23056ab62 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:37.974275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520782d5-79b8-4c89-b16e-61e23056ab62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:37.982891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15d184-8e27-4b34-bd2c-a8a167455567 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:37.996101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-520782d5-79b8-4c89-b16e-61e23056ab62 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:37.997439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-520782d5-79b8-4c89-b16e-61e23056ab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:38.038672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:38.039205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:38.039510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:38.039777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:38.042239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:38.083689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1144/2283] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:38.103006 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520782d5-79b8-4c89-b16e-61e23056ab62 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:38.107088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-520782d5-79b8-4c89-b16e-61e23056ab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:38.118828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:38.148487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:38.151441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:38.163042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1140/2284] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:38.222620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520782d5-79b8-4c89-b16e-61e23056ab62 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:38.240415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf6064-3e09-45d5-914f-fd73ba31ee08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:38.241165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6064-3e09-45d5-914f-fd73ba31ee08 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:38.241512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6064-3e09-45d5-914f-fd73ba31ee08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:38.241787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6064-3e09-45d5-914f-fd73ba31ee08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:38.257170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf6064-3e09-45d5-914f-fd73ba31ee08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:38.259876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf6064-3e09-45d5-914f-fd73ba31ee08 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:38.272641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1145/2285] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:38.598546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:38.600465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:38.618476 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc54d7-5125-4484-b5ca-13af00d4dcef service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:38.625668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:38.626327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:38.626636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:38.626864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] PATH_INFO: `/v3/projects/c94606bf6d564285965b9d0256de535c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:38.628048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94606bf6d564285965b9d0256de535c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:38.630083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:38.638444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:38.698810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94606bf6d564285965b9d0256de535c', 'request_id': 'req-547909d4-70bd-4251-85ff-bcc3eb927c0f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:38.705725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94606bf6d564285965b9d0256de535c', 'request_id': 'req-547909d4-70bd-4251-85ff-bcc3eb927c0f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:38.714189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94606bf6d564285965b9d0256de535c', 'request_id': 'req-547909d4-70bd-4251-85ff-bcc3eb927c0f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:38.731914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] The token cache is being invalidate because project c94606bf6d564285965b9d0256de535c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:38.732313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:38.735273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:38.740019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1141/2286] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:38 2022] DELETE /identity/v3/projects/c94606bf6d564285965b9d0256de535c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:38.745150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf6064-3e09-45d5-914f-fd73ba31ee08 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:38.757427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf6064-3e09-45d5-914f-fd73ba31ee08 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:38.872255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:38.872922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:38.873177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:38.873578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] PATH_INFO: `/v3/users/8034f1ec47824eda9e4862fec718254f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:38.875944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8034f1ec47824eda9e4862fec718254f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:38.879559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:38.977959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8034f1ec47824eda9e4862fec718254f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:38.978724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8034f1ec47824eda9e4862fec718254f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:38.994880 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8034f1ec47824eda9e4862fec718254f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:38.995516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8034f1ec47824eda9e4862fec718254f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:39.015087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0c7565dd-f03d-4750-9c90-1de5096ce793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8034f1ec47824eda9e4862fec718254f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:39.036031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1146/2287] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:38 2022] DELETE /identity/v3/users/8034f1ec47824eda9e4862fec718254f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:39.401520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:39.403628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:39.435679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547909d4-70bd-4251-85ff-bcc3eb927c0f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:39.518125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:39.519020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:39.519595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:39.520053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] PATH_INFO: `/v3/projects/118f7575965d4102b243c7ffc702efc9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:39.521357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] RBAC: Authorizing `identity:delete_project(project_id=118f7575965d4102b243c7ffc702efc9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:39.523540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:39.610353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118f7575965d4102b243c7ffc702efc9', 'request_id': 'req-e5624cb1-0b83-48e5-a111-c063dffc709c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:39.623519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118f7575965d4102b243c7ffc702efc9', 'request_id': 'req-e5624cb1-0b83-48e5-a111-c063dffc709c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:39.630436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118f7575965d4102b243c7ffc702efc9', 'request_id': 'req-e5624cb1-0b83-48e5-a111-c063dffc709c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:39.659545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] The token cache is being invalidate because project 118f7575965d4102b243c7ffc702efc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:39.659545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:39.659545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5624cb1-0b83-48e5-a111-c063dffc709c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:39.660438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1142/2288] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:39 2022] DELETE /identity/v3/projects/118f7575965d4102b243c7ffc702efc9 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:39.665831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdf45d-5eef-456d-bf1d-24603254de44 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:39.667128 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdf45d-5eef-456d-bf1d-24603254de44 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:39.667643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdf45d-5eef-456d-bf1d-24603254de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:39.700863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4bfdf45d-5eef-456d-bf1d-24603254de44 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:39.702719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4bfdf45d-5eef-456d-bf1d-24603254de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:39.726661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:39.771606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1147/2289] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:39.784338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97378730-93e9-40d9-b5bb-00e0cd7ad0c1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:39.784338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97378730-93e9-40d9-b5bb-00e0cd7ad0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:39.784338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97378730-93e9-40d9-b5bb-00e0cd7ad0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:39.810334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-97378730-93e9-40d9-b5bb-00e0cd7ad0c1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:39.810334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-97378730-93e9-40d9-b5bb-00e0cd7ad0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:39.817638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdf45d-5eef-456d-bf1d-24603254de44 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:39.818744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1143/2290] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:39.819408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4bfdf45d-5eef-456d-bf1d-24603254de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:39.824203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97378730-93e9-40d9-b5bb-00e0cd7ad0c1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:39.834487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97378730-93e9-40d9-b5bb-00e0cd7ad0c1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:39.840082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:39.840746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:39.841128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:39.841458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:39.842189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:39.842488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:39.844251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:39.857928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1144/2291] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:39.864052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:39.877774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:39.882542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b4e8d-c5a2-44fb-b5e3-d8f28f88f8cd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:39.891039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc024f6-12b2-4a18-95b9-ea2197b77ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:39.891731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc024f6-12b2-4a18-95b9-ea2197b77ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:39.891983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc024f6-12b2-4a18-95b9-ea2197b77ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:39.892254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc024f6-12b2-4a18-95b9-ea2197b77ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:39.892891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc024f6-12b2-4a18-95b9-ea2197b77ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:39.895196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc024f6-12b2-4a18-95b9-ea2197b77ec4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:39.897491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdf45d-5eef-456d-bf1d-24603254de44 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:39.902147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:39.902997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:39.903484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:39.903957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:39.908055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:39.943847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1145/2292] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:39 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:39.944586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc024f6-12b2-4a18-95b9-ea2197b77ec4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:39.992762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:39.996611 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.021124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1148/2293] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:40.046950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc024f6-12b2-4a18-95b9-ea2197b77ec4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.052356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a714af8-9af4-4b8b-8f61-1cd827603d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.053115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a714af8-9af4-4b8b-8f61-1cd827603d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.053393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a714af8-9af4-4b8b-8f61-1cd827603d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.053661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a714af8-9af4-4b8b-8f61-1cd827603d33 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.054401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a714af8-9af4-4b8b-8f61-1cd827603d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.056436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a714af8-9af4-4b8b-8f61-1cd827603d33 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.086327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1146/2294] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:39 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:40.091509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.107644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da69eb-622c-4fab-9df4-30b75e7f9ccf service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.114426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0e20e-d121-460b-a040-d9ed374662c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.115064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e20e-d121-460b-a040-d9ed374662c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.115450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e20e-d121-460b-a040-d9ed374662c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.115799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0e20e-d121-460b-a040-d9ed374662c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.116566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0e20e-d121-460b-a040-d9ed374662c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.119743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0e20e-d121-460b-a040-d9ed374662c2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.147755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1149/2295] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:20:40.151881 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a714af8-9af4-4b8b-8f61-1cd827603d33 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.165940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a714af8-9af4-4b8b-8f61-1cd827603d33 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.172426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b7e47-c35d-4549-9e4b-f8df358f959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.173030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b7e47-c35d-4549-9e4b-f8df358f959b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.173304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b7e47-c35d-4549-9e4b-f8df358f959b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.173755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b7e47-c35d-4549-9e4b-f8df358f959b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.174507 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b7e47-c35d-4549-9e4b-f8df358f959b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.176163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b7e47-c35d-4549-9e4b-f8df358f959b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.189596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1147/2296] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:40.193612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0e20e-d121-460b-a040-d9ed374662c2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.206814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0e20e-d121-460b-a040-d9ed374662c2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.216217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.217172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.217779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.218107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] PATH_INFO: `/v3/projects/d474628b33844c52a62067b846127022/users/a08e533f5c4649fe8f40bfb6b6f3a032/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.237557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d474628b33844c52a62067b846127022, user_id=a08e533f5c4649fe8f40bfb6b6f3a032, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.240456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.261369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1150/2297] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:40 2022] PUT /identity/v3/projects/d474628b33844c52a62067b846127022/users/a08e533f5c4649fe8f40bfb6b6f3a032/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:40.272471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadb50c-2072-444e-9d31-0b2781318431 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.272819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadb50c-2072-444e-9d31-0b2781318431 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.273037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadb50c-2072-444e-9d31-0b2781318431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.313589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ceadb50c-2072-444e-9d31-0b2781318431 None None] MFA Rules not processed for user `a08e533f5c4649fe8f40bfb6b6f3a032`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:40.314871 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ceadb50c-2072-444e-9d31-0b2781318431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:40.361175 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.365134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:40.425811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1148/2298] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:40.427369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadb50c-2072-444e-9d31-0b2781318431 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.439625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:40.467362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604110a8-6798-4d86-a715-a510d1ceb92e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.472839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.475297 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.475943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.476453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] PATH_INFO: `/v3/users/ce19853819814ed9906aa9b5cae809cb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.479517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce19853819814ed9906aa9b5cae809cb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.483208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.500795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:40.517823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadb50c-2072-444e-9d31-0b2781318431 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.523052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.523704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.523980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.524412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.525056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.527144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.553327 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce19853819814ed9906aa9b5cae809cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:40.554487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce19853819814ed9906aa9b5cae809cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:40.565935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce19853819814ed9906aa9b5cae809cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:40.566818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce19853819814ed9906aa9b5cae809cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:40.567751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1149/2299] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:40 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:20:40.572063 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.574446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:40.584475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde15f19-0e94-4bb8-968c-a7e7daa94539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce19853819814ed9906aa9b5cae809cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:40.600195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1151/2300] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:40 2022] DELETE /identity/v3/users/ce19853819814ed9906aa9b5cae809cb => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:40.604659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1690a-9b60-4677-a40c-6dab0a5c93df None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.604941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1690a-9b60-4677-a40c-6dab0a5c93df None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.605212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1690a-9b60-4677-a40c-6dab0a5c93df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.643185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d2f1690a-9b60-4677-a40c-6dab0a5c93df None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:40.644728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d2f1690a-9b60-4677-a40c-6dab0a5c93df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:40.668741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b71e5a-213f-440d-afca-3aea01b36efb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.748853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5fdd9-b8fd-45c2-8b56-c65fd39b1857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.749549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fdd9-b8fd-45c2-8b56-c65fd39b1857 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.750542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fdd9-b8fd-45c2-8b56-c65fd39b1857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.750654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1152/2301] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:40.750737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fdd9-b8fd-45c2-8b56-c65fd39b1857 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.751534 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5fdd9-b8fd-45c2-8b56-c65fd39b1857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.761526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1690a-9b60-4677-a40c-6dab0a5c93df None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.761718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5fdd9-b8fd-45c2-8b56-c65fd39b1857 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.763079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d2f1690a-9b60-4677-a40c-6dab0a5c93df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:40.793175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1150/2302] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:40 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:40.798461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5fdd9-b8fd-45c2-8b56-c65fd39b1857 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.823822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5fdd9-b8fd-45c2-8b56-c65fd39b1857 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.856993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.857627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.857902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.858171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.858825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.862567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.867788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1690a-9b60-4677-a40c-6dab0a5c93df None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.878117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1151/2303] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:40.879014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.881472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:40.883760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719d96b-4a0e-4f9c-a915-c8fd9e8e70fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.884518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719d96b-4a0e-4f9c-a915-c8fd9e8e70fd service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.884871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719d96b-4a0e-4f9c-a915-c8fd9e8e70fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.885172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719d96b-4a0e-4f9c-a915-c8fd9e8e70fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.906985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719d96b-4a0e-4f9c-a915-c8fd9e8e70fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.908700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719d96b-4a0e-4f9c-a915-c8fd9e8e70fd service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:40.920366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1153/2304] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:40.921530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719d96b-4a0e-4f9c-a915-c8fd9e8e70fd service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:40.941124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719d96b-4a0e-4f9c-a915-c8fd9e8e70fd service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:40.957561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90807-3d61-4b55-9e7f-d175382c3d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:40.958248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90807-3d61-4b55-9e7f-d175382c3d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:40.958531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90807-3d61-4b55-9e7f-d175382c3d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:40.958920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90807-3d61-4b55-9e7f-d175382c3d4b admin admin] PATH_INFO: `/v3/projects/d4c99fbf68b849359f2222285d2b7c4a/users/229040914d364c7faa5764dc8ca815ca/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:40.987371 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:40.992260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf90807-3d61-4b55-9e7f-d175382c3d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c99fbf68b849359f2222285d2b7c4a, user_id=229040914d364c7faa5764dc8ca815ca, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:40.996452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf90807-3d61-4b55-9e7f-d175382c3d4b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:41.020747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade400b2-38ac-43cd-8438-0231b9f60229 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:41.024731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1154/2305] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:40 2022] PUT /identity/v3/projects/d4c99fbf68b849359f2222285d2b7c4a/users/229040914d364c7faa5764dc8ca815ca/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:41.031528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90807-3d61-4b55-9e7f-d175382c3d4b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:41.048697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:41.140308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:41.142605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:41.144266 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:41.144759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:41.146345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90807-3d61-4b55-9e7f-d175382c3d4b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:41.147566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:41.152741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dc8d3-d4b0-405b-a3fc-3ae2a00d43bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:41.153430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dc8d3-d4b0-405b-a3fc-3ae2a00d43bd admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:41.153711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dc8d3-d4b0-405b-a3fc-3ae2a00d43bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:41.153979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dc8d3-d4b0-405b-a3fc-3ae2a00d43bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:41.154640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dc8d3-d4b0-405b-a3fc-3ae2a00d43bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:41.156518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dc8d3-d4b0-405b-a3fc-3ae2a00d43bd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:41.171746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1155/2306] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:41.176809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dc8d3-d4b0-405b-a3fc-3ae2a00d43bd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:41.193316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dc8d3-d4b0-405b-a3fc-3ae2a00d43bd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:41.199477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a062ef0-b1bc-4bad-a22c-cb996ee93d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:41.200444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062ef0-b1bc-4bad-a22c-cb996ee93d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:41.200825 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062ef0-b1bc-4bad-a22c-cb996ee93d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:41.201186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062ef0-b1bc-4bad-a22c-cb996ee93d8b admin admin] PATH_INFO: `/v3/projects/d4c99fbf68b849359f2222285d2b7c4a/users/229040914d364c7faa5764dc8ca815ca/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:41.209819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a062ef0-b1bc-4bad-a22c-cb996ee93d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c99fbf68b849359f2222285d2b7c4a, user_id=229040914d364c7faa5764dc8ca815ca, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:41.214066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a062ef0-b1bc-4bad-a22c-cb996ee93d8b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:41.241248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1156/2307] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:41 2022] PUT /identity/v3/projects/d4c99fbf68b849359f2222285d2b7c4a/users/229040914d364c7faa5764dc8ca815ca/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:41.242942 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a037d6-f028-4b87-b488-2d2df3e29ed1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:41.243446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a037d6-f028-4b87-b488-2d2df3e29ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:41.243859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a037d6-f028-4b87-b488-2d2df3e29ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:41.261125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:41.262822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:41.276681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1152/2308] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:41.285446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-68a037d6-f028-4b87-b488-2d2df3e29ed1 None None] MFA Rules not processed for user `229040914d364c7faa5764dc8ca815ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:41.286683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-68a037d6-f028-4b87-b488-2d2df3e29ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:41.394515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1157/2309] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:41.449319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:41.458370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:41.490164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d9ba7-60bd-4b87-86d2-6dc9726e7a32 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:41.608896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a037d6-f028-4b87-b488-2d2df3e29ed1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:41.613514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-68a037d6-f028-4b87-b488-2d2df3e29ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:41.616931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087390-7e84-495f-b0f9-fb56b92737b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:41.617582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087390-7e84-495f-b0f9-fb56b92737b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:41.618176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087390-7e84-495f-b0f9-fb56b92737b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:41.618498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087390-7e84-495f-b0f9-fb56b92737b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:41.632982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087390-7e84-495f-b0f9-fb56b92737b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:41.635040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087390-7e84-495f-b0f9-fb56b92737b3 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:41.649143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1153/2310] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:41.705453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:41.736706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a037d6-f028-4b87-b488-2d2df3e29ed1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:41.746158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:41.747394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:41.748186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:41.748631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] PATH_INFO: `/v3/projects/a8426753141444c581bd6cecf00b5221` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:41.750389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8426753141444c581bd6cecf00b5221)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:41.753076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:41.844713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8426753141444c581bd6cecf00b5221', 'request_id': 'req-29c999d2-9634-419f-bcc6-f33df035250a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:41.851274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8426753141444c581bd6cecf00b5221', 'request_id': 'req-29c999d2-9634-419f-bcc6-f33df035250a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:41.860669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8426753141444c581bd6cecf00b5221', 'request_id': 'req-29c999d2-9634-419f-bcc6-f33df035250a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:41.887296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] The token cache is being invalidate because project a8426753141444c581bd6cecf00b5221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:41.887668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:41.888571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:41.896536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1158/2311] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:41 2022] DELETE /identity/v3/projects/a8426753141444c581bd6cecf00b5221 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:41.900324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087390-7e84-495f-b0f9-fb56b92737b3 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:41.918455 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087390-7e84-495f-b0f9-fb56b92737b3 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:42.015936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:42.017019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:42.017362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:42.017732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] PATH_INFO: `/v3/users/fae8adf8c2a742a3a603cb7483f81914` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:42.019869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae8adf8c2a742a3a603cb7483f81914)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:42.021409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:42.042430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:42.099504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae8adf8c2a742a3a603cb7483f81914'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.100439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae8adf8c2a742a3a603cb7483f81914'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.120894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae8adf8c2a742a3a603cb7483f81914'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.139665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae8adf8c2a742a3a603cb7483f81914'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.153033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f2b79f9e-342e-457b-bd4a-43bd3dfb22ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae8adf8c2a742a3a603cb7483f81914'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.160460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1154/2312] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:41 2022] DELETE /identity/v3/users/fae8adf8c2a742a3a603cb7483f81914 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:42.253119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:42.258255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:42.275751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c999d2-9634-419f-bcc6-f33df035250a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:42.351304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:42.352269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:42.352714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:42.352988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] PATH_INFO: `/v3/users/a08e533f5c4649fe8f40bfb6b6f3a032` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:42.354580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08e533f5c4649fe8f40bfb6b6f3a032)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:42.356721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:42.434400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08e533f5c4649fe8f40bfb6b6f3a032'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.435119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08e533f5c4649fe8f40bfb6b6f3a032'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.448276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08e533f5c4649fe8f40bfb6b6f3a032'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.448907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08e533f5c4649fe8f40bfb6b6f3a032'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.482146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08e533f5c4649fe8f40bfb6b6f3a032'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.512733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1159/2313] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:42 2022] DELETE /identity/v3/users/a08e533f5c4649fe8f40bfb6b6f3a032 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:42.516508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f676a2a-0ce2-4759-9eb9-a200b63aee8a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:42.516792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f676a2a-0ce2-4759-9eb9-a200b63aee8a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:42.517065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f676a2a-0ce2-4759-9eb9-a200b63aee8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:42.540645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4f676a2a-0ce2-4759-9eb9-a200b63aee8a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:42.542336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4f676a2a-0ce2-4759-9eb9-a200b63aee8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:42.583115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:42.584709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:42.593053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:42.686614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1155/2314] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:42.702892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f676a2a-0ce2-4759-9eb9-a200b63aee8a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:42.704677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4f676a2a-0ce2-4759-9eb9-a200b63aee8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:42.749029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf4f19-d32a-4695-b28a-10db1054f49c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:42.760360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:42.762403 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:42.762976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:42.763878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] PATH_INFO: `/v3/projects/530f7f7e1e7a42fdbaea8ea77f3ee70c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:42.765602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530f7f7e1e7a42fdbaea8ea77f3ee70c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:42.769340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:42.790535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f676a2a-0ce2-4759-9eb9-a200b63aee8a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:42.796191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d269de-173d-41fd-a765-efa855e11912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:42.796965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d269de-173d-41fd-a765-efa855e11912 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:42.797253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d269de-173d-41fd-a765-efa855e11912 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:42.797522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d269de-173d-41fd-a765-efa855e11912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:42.809709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d269de-173d-41fd-a765-efa855e11912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:42.811913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d269de-173d-41fd-a765-efa855e11912 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:42.824388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1156/2315] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:42.859954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530f7f7e1e7a42fdbaea8ea77f3ee70c', 'request_id': 'req-607b2986-24e8-4ee5-93ac-b7adf2aaf607'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.879443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530f7f7e1e7a42fdbaea8ea77f3ee70c', 'request_id': 'req-607b2986-24e8-4ee5-93ac-b7adf2aaf607'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.905551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530f7f7e1e7a42fdbaea8ea77f3ee70c', 'request_id': 'req-607b2986-24e8-4ee5-93ac-b7adf2aaf607'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.946561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] The token cache is being invalidate because project 530f7f7e1e7a42fdbaea8ea77f3ee70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:42.947492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.949016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:42.951471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1160/2316] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:42 2022] DELETE /identity/v3/projects/530f7f7e1e7a42fdbaea8ea77f3ee70c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:42.956843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d269de-173d-41fd-a765-efa855e11912 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:42.968173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d269de-173d-41fd-a765-efa855e11912 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:43.049628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:43.108006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:43.108657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:43.109125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:43.109352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] PATH_INFO: `/v3/users/22ba02bfbf96428fbd8b8a55531c0a87` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:43.119967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22ba02bfbf96428fbd8b8a55531c0a87)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:43.122132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:43.254123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22ba02bfbf96428fbd8b8a55531c0a87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.257046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22ba02bfbf96428fbd8b8a55531c0a87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.278714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22ba02bfbf96428fbd8b8a55531c0a87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.290376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22ba02bfbf96428fbd8b8a55531c0a87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.303730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22ba02bfbf96428fbd8b8a55531c0a87'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.319862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1157/2317] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:42 2022] DELETE /identity/v3/users/22ba02bfbf96428fbd8b8a55531c0a87 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:43.443077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:43.444700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:43.531814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b2986-24e8-4ee5-93ac-b7adf2aaf607 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:43.618122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:43.618764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:43.619106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:43.619421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] PATH_INFO: `/v3/projects/d474628b33844c52a62067b846127022` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:43.620633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d474628b33844c52a62067b846127022)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:43.625511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:43.673399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:43.695078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd474628b33844c52a62067b846127022', 'request_id': 'req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.707389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd474628b33844c52a62067b846127022', 'request_id': 'req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.721716 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd474628b33844c52a62067b846127022', 'request_id': 'req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.742312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] The token cache is being invalidate because project d474628b33844c52a62067b846127022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:43.743215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.744498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.747080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1161/2318] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:43 2022] DELETE /identity/v3/projects/d474628b33844c52a62067b846127022 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:43.752190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:43.754408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:43.778555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5b350-7880-402a-8b9e-31767a9768b7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:43.853822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:43.854619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:43.854890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:43.855170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] PATH_INFO: `/v3/users/229040914d364c7faa5764dc8ca815ca` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:43.856394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=229040914d364c7faa5764dc8ca815ca)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:43.858367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:43.936470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229040914d364c7faa5764dc8ca815ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.937153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229040914d364c7faa5764dc8ca815ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.942853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:43.956620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229040914d364c7faa5764dc8ca815ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.968092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229040914d364c7faa5764dc8ca815ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.977761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b748efb-26f1-474a-91c9-b3caf7e0e2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229040914d364c7faa5764dc8ca815ca'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:43.985484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1158/2319] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:43 2022] DELETE /identity/v3/users/229040914d364c7faa5764dc8ca815ca => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:44.062694 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:44.065590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:44.130150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf0d4-7b83-4da5-a216-5d7ef8a6da65 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:44.217893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:44.218604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:44.219015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:44.219948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] PATH_INFO: `/v3/projects/abc088805cc84f62978b1b0a102f1bc2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:44.220656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abc088805cc84f62978b1b0a102f1bc2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:44.222938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:44.288836 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abc088805cc84f62978b1b0a102f1bc2', 'request_id': 'req-e2d8da39-df91-41e9-b6e4-0118e9d67be9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.300210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abc088805cc84f62978b1b0a102f1bc2', 'request_id': 'req-e2d8da39-df91-41e9-b6e4-0118e9d67be9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.310539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abc088805cc84f62978b1b0a102f1bc2', 'request_id': 'req-e2d8da39-df91-41e9-b6e4-0118e9d67be9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.330941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] The token cache is being invalidate because project abc088805cc84f62978b1b0a102f1bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:44.330941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.333053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.338865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1162/2320] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:44 2022] DELETE /identity/v3/projects/abc088805cc84f62978b1b0a102f1bc2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:44.341631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ef3c-8398-49ab-8fd8-3329086d53d8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:44.341911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ef3c-8398-49ab-8fd8-3329086d53d8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:44.343516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ef3c-8398-49ab-8fd8-3329086d53d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:44.386018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-66c6ef3c-8398-49ab-8fd8-3329086d53d8 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:44.387830 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66c6ef3c-8398-49ab-8fd8-3329086d53d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:44.449266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:44.453134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:44.552872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1159/2321] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:44.562061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c2707-34eb-42cd-9934-8bc39f40d524 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:44.562294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c2707-34eb-42cd-9934-8bc39f40d524 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:44.562810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c2707-34eb-42cd-9934-8bc39f40d524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:44.572244 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:44.593798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c26c2707-34eb-42cd-9934-8bc39f40d524 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:44.594805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8da39-df91-41e9-b6e4-0118e9d67be9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:44.595361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c26c2707-34eb-42cd-9934-8bc39f40d524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:44.601426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:44.602632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:44.602944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:44.604517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] PATH_INFO: `/v3/projects/d4c99fbf68b849359f2222285d2b7c4a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:44.606492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c99fbf68b849359f2222285d2b7c4a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:44.607602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1160/2322] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:44.608740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:44.611948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c2707-34eb-42cd-9934-8bc39f40d524 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:44.623286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c2707-34eb-42cd-9934-8bc39f40d524 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:44.630773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:44.631726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:44.632026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:44.632535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:44.633357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:44.633911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:44.636697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:44.655869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1161/2323] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:20:44.660492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:44.675962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc61482-d169-41cb-b2d4-5e7d36fa9d56 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:44.690210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c99fbf68b849359f2222285d2b7c4a', 'request_id': 'req-5226496f-27a2-49f7-89bd-c0b548ecb07c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.702531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c99fbf68b849359f2222285d2b7c4a', 'request_id': 'req-5226496f-27a2-49f7-89bd-c0b548ecb07c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.714834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c99fbf68b849359f2222285d2b7c4a', 'request_id': 'req-5226496f-27a2-49f7-89bd-c0b548ecb07c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.744891 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] The token cache is being invalidate because project d4c99fbf68b849359f2222285d2b7c4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:44.745380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.747281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5226496f-27a2-49f7-89bd-c0b548ecb07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:44.750930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1163/2324] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:44 2022] DELETE /identity/v3/projects/d4c99fbf68b849359f2222285d2b7c4a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:44.757822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a0b0b-04c4-419d-85d5-9f7de37701ed None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:44.758200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a0b0b-04c4-419d-85d5-9f7de37701ed None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:44.758918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a0b0b-04c4-419d-85d5-9f7de37701ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:44.780237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:44.781164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:44.781508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:44.781785 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:44.782638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:44.784815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:44.792029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3a7a0b0b-04c4-419d-85d5-9f7de37701ed None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:44.794834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3a7a0b0b-04c4-419d-85d5-9f7de37701ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:44.821335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1164/2325] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:44.833262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c41189-c635-4917-a0e7-fe777972de81 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:44.834679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c41189-c635-4917-a0e7-fe777972de81 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:44.835174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c41189-c635-4917-a0e7-fe777972de81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:44.862927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1162/2326] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:44 2022] POST /identity/v3/projects => generated 391 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:20:44.864174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:44.866646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:44.867745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-95c41189-c635-4917-a0e7-fe777972de81 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:44.869274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-95c41189-c635-4917-a0e7-fe777972de81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:44.978165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:45.011407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1165/2327] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:45.015538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c41189-c635-4917-a0e7-fe777972de81 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.031547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecac9f-3fac-4350-911d-273b0d6a9e0e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.032166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c41189-c635-4917-a0e7-fe777972de81 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.039046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72919e5d-7229-471b-9cc3-ecf76c0f53fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.039989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72919e5d-7229-471b-9cc3-ecf76c0f53fb admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.040478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72919e5d-7229-471b-9cc3-ecf76c0f53fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.040790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.041291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72919e5d-7229-471b-9cc3-ecf76c0f53fb admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.041872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.042321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.042762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72919e5d-7229-471b-9cc3-ecf76c0f53fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.042963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.043993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:45.044513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.046254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72919e5d-7229-471b-9cc3-ecf76c0f53fb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.048419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.071149 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1166/2328] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:20:45.075615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1163/2329] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:44 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:20:45.076737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.079996 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72919e5d-7229-471b-9cc3-ecf76c0f53fb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.093260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72919e5d-7229-471b-9cc3-ecf76c0f53fb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.101820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2df62c-8c44-45c7-94e3-5e12b2573344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.103417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2df62c-8c44-45c7-94e3-5e12b2573344 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.104128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2df62c-8c44-45c7-94e3-5e12b2573344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.104943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2df62c-8c44-45c7-94e3-5e12b2573344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.105955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2df62c-8c44-45c7-94e3-5e12b2573344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.106087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7478dc-82a6-45a3-9c03-9f4579a94d5e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.108328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2df62c-8c44-45c7-94e3-5e12b2573344 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.111915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ba213-715f-42fe-a7c0-6a9638fcd7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.112657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ba213-715f-42fe-a7c0-6a9638fcd7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.112994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ba213-715f-42fe-a7c0-6a9638fcd7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.113343 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ba213-715f-42fe-a7c0-6a9638fcd7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.114051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ba213-715f-42fe-a7c0-6a9638fcd7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.116627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ba213-715f-42fe-a7c0-6a9638fcd7ce admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.123936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1164/2330] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:45.128099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2df62c-8c44-45c7-94e3-5e12b2573344 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.146341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2df62c-8c44-45c7-94e3-5e12b2573344 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.152473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157f8f1-3548-4bc0-bd6e-af1dcb7b671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.153102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157f8f1-3548-4bc0-bd6e-af1dcb7b671f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.153399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157f8f1-3548-4bc0-bd6e-af1dcb7b671f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.154119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157f8f1-3548-4bc0-bd6e-af1dcb7b671f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.155617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157f8f1-3548-4bc0-bd6e-af1dcb7b671f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.157278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157f8f1-3548-4bc0-bd6e-af1dcb7b671f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.163605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1167/2331] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:45 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:45.165709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ba213-715f-42fe-a7c0-6a9638fcd7ce admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.179525 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1165/2332] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:20:45.181708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157f8f1-3548-4bc0-bd6e-af1dcb7b671f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.209957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157f8f1-3548-4bc0-bd6e-af1dcb7b671f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.283220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:45.331447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ba213-715f-42fe-a7c0-6a9638fcd7ce admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.344622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dcc86-69a2-4804-b17b-16245d60f834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.345594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dcc86-69a2-4804-b17b-16245d60f834 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.346385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51370cb7-4de9-4b47-ba73-6c1a40f84f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.346769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dcc86-69a2-4804-b17b-16245d60f834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.347171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370cb7-4de9-4b47-ba73-6c1a40f84f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.347317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dcc86-69a2-4804-b17b-16245d60f834 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.347502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370cb7-4de9-4b47-ba73-6c1a40f84f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.347886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370cb7-4de9-4b47-ba73-6c1a40f84f71 admin admin] PATH_INFO: `/v3/projects/7d5986426ceb4864bd44f0f60b95933f/users/045a25aae6164763860a0227f5ea5bce/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.348368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dcc86-69a2-4804-b17b-16245d60f834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.354297 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dcc86-69a2-4804-b17b-16245d60f834 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.394072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1168/2333] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:45 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:20:45.394440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51370cb7-4de9-4b47-ba73-6c1a40f84f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5986426ceb4864bd44f0f60b95933f, user_id=045a25aae6164763860a0227f5ea5bce, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.398866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dcc86-69a2-4804-b17b-16245d60f834 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.400942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51370cb7-4de9-4b47-ba73-6c1a40f84f71 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.415511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dcc86-69a2-4804-b17b-16245d60f834 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.421774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1166/2334] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:45 2022] PUT /identity/v3/projects/7d5986426ceb4864bd44f0f60b95933f/users/045a25aae6164763860a0227f5ea5bce/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:45.492646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70810c-5001-4ce1-9e05-ffc35d255297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.493452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70810c-5001-4ce1-9e05-ffc35d255297 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.494091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70810c-5001-4ce1-9e05-ffc35d255297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.494931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70810c-5001-4ce1-9e05-ffc35d255297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.495917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70810c-5001-4ce1-9e05-ffc35d255297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.497271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323d027-f364-4f1c-beaa-9c5208e79105 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.497551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323d027-f364-4f1c-beaa-9c5208e79105 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.497667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70810c-5001-4ce1-9e05-ffc35d255297 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.497818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323d027-f364-4f1c-beaa-9c5208e79105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.517265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1169/2335] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:45.518637 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70810c-5001-4ce1-9e05-ffc35d255297 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.530389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70810c-5001-4ce1-9e05-ffc35d255297 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.535078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c323d027-f364-4f1c-beaa-9c5208e79105 None None] MFA Rules not processed for user `045a25aae6164763860a0227f5ea5bce`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:45.536005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820c60a-a470-4631-a543-df4be2362e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.536627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c323d027-f364-4f1c-beaa-9c5208e79105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:45.536755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820c60a-a470-4631-a543-df4be2362e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.536972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820c60a-a470-4631-a543-df4be2362e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.537184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820c60a-a470-4631-a543-df4be2362e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.537833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820c60a-a470-4631-a543-df4be2362e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.539705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820c60a-a470-4631-a543-df4be2362e37 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.556554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1170/2336] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:45.561052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820c60a-a470-4631-a543-df4be2362e37 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.571490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820c60a-a470-4631-a543-df4be2362e37 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.577729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5098e5a-d33a-4c7d-be36-bd8c7ddbd25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.578408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098e5a-d33a-4c7d-be36-bd8c7ddbd25f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.578684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098e5a-d33a-4c7d-be36-bd8c7ddbd25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.578954 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098e5a-d33a-4c7d-be36-bd8c7ddbd25f admin admin] PATH_INFO: `/v3/projects/570d114f4722462e8dc8d51be8b69c3d/users/e736dfaf2c704ef2986f9880503876be/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.596317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5098e5a-d33a-4c7d-be36-bd8c7ddbd25f admin admin] RBAC: Authorizing `identity:create_grant(project_id=570d114f4722462e8dc8d51be8b69c3d, user_id=e736dfaf2c704ef2986f9880503876be, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.599177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5098e5a-d33a-4c7d-be36-bd8c7ddbd25f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.614933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1171/2337] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:45 2022] PUT /identity/v3/projects/570d114f4722462e8dc8d51be8b69c3d/users/e736dfaf2c704ef2986f9880503876be/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:45.626632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c145-03cb-4db2-b4da-21672063955e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.626895 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c145-03cb-4db2-b4da-21672063955e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.627177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c145-03cb-4db2-b4da-21672063955e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.640049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1167/2338] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:45.654124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323d027-f364-4f1c-beaa-9c5208e79105 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.655971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c323d027-f364-4f1c-beaa-9c5208e79105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:45.664947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fdd2c145-03cb-4db2-b4da-21672063955e None None] MFA Rules not processed for user `e736dfaf2c704ef2986f9880503876be`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:45.667511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fdd2c145-03cb-4db2-b4da-21672063955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:45.679862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:45.736258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323d027-f364-4f1c-beaa-9c5208e79105 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.765598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1172/2339] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:45.788674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2c145-03cb-4db2-b4da-21672063955e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.790470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fdd2c145-03cb-4db2-b4da-21672063955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:45.811103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.811759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.812038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.812309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.824413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:45.885168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2c145-03cb-4db2-b4da-21672063955e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.888662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.890281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.891521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.892303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.892658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.893050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] PATH_INFO: `/v3/users/a560c330d47d440aa18214c330b1a7cb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.894368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a560c330d47d440aa18214c330b1a7cb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.896458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.902411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1168/2340] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:45.903096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:45.904430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:45.922053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca67841-83a0-42d0-9eae-e9d2092ca4cf service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:45.930119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba3ef-5f6e-4df8-b27e-ea4820ccd6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:45.931868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba3ef-5f6e-4df8-b27e-ea4820ccd6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:45.932511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba3ef-5f6e-4df8-b27e-ea4820ccd6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:45.933094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba3ef-5f6e-4df8-b27e-ea4820ccd6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:45.948906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dba3ef-5f6e-4df8-b27e-ea4820ccd6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:45.953983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dba3ef-5f6e-4df8-b27e-ea4820ccd6a4 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:45.966783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a560c330d47d440aa18214c330b1a7cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:45.967528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a560c330d47d440aa18214c330b1a7cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:45.970201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1169/2341] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:45.978071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a560c330d47d440aa18214c330b1a7cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:45.978928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a560c330d47d440aa18214c330b1a7cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.000774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a560c330d47d440aa18214c330b1a7cb'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.024618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1173/2342] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:45 2022] DELETE /identity/v3/users/a560c330d47d440aa18214c330b1a7cb => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:46.030902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ac80-6055-4100-8880-613861eace38 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:46.031460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ac80-6055-4100-8880-613861eace38 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:46.031821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ac80-6055-4100-8880-613861eace38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:46.060478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ca26ac80-6055-4100-8880-613861eace38 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:46.061705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ca26ac80-6055-4100-8880-613861eace38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:46.165809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1170/2343] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:46.176784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:46.178645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:46.253301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d83eb-a99e-4501-bbd3-7adeca544961 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:46.257389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcd7d6-b10a-45cc-afb1-48a91420c564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:46.258121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd7d6-b10a-45cc-afb1-48a91420c564 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:46.258842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd7d6-b10a-45cc-afb1-48a91420c564 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:46.259524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd7d6-b10a-45cc-afb1-48a91420c564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:46.283062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcd7d6-b10a-45cc-afb1-48a91420c564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:46.287588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcd7d6-b10a-45cc-afb1-48a91420c564 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:46.316171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1174/2344] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:46.497152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26ac80-6055-4100-8880-613861eace38 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:46.497503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ca26ac80-6055-4100-8880-613861eace38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:46.519013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26ac80-6055-4100-8880-613861eace38 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:46.526339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:46.527064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:46.527372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:46.527757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] PATH_INFO: `/v3/users/be5436999ce84450ac2557fc7110e0a1` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:46.531977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5436999ce84450ac2557fc7110e0a1)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:46.535628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:46.572803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:46.635263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5436999ce84450ac2557fc7110e0a1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.636154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5436999ce84450ac2557fc7110e0a1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.652778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5436999ce84450ac2557fc7110e0a1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.664369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5436999ce84450ac2557fc7110e0a1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.687946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5436999ce84450ac2557fc7110e0a1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.696612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1171/2345] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:46 2022] DELETE /identity/v3/users/be5436999ce84450ac2557fc7110e0a1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:46.702223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0dcd6-e177-473e-8503-4b0ec682a0c9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:46.702731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0dcd6-e177-473e-8503-4b0ec682a0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:46.703095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0dcd6-e177-473e-8503-4b0ec682a0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:46.706836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:46.708602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:46.746698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3af0dcd6-e177-473e-8503-4b0ec682a0c9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:46.748064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3af0dcd6-e177-473e-8503-4b0ec682a0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:46.792364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707aa1e-deaf-4c5a-a612-d827aea0deb4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:46.797542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:46.798382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:46.798697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:46.798976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] PATH_INFO: `/v3/projects/f2ffdb40e3f846e99cd245544b9aeebd` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:46.800265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2ffdb40e3f846e99cd245544b9aeebd)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:46.802786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:46.804113 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:46.837979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1175/2346] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:46.849682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0dcd6-e177-473e-8503-4b0ec682a0c9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:46.851567 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3af0dcd6-e177-473e-8503-4b0ec682a0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:46.859161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ffdb40e3f846e99cd245544b9aeebd', 'request_id': 'req-1b84e53e-7f7a-4f47-972f-c191023c0dd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.865759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ffdb40e3f846e99cd245544b9aeebd', 'request_id': 'req-1b84e53e-7f7a-4f47-972f-c191023c0dd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.880228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ffdb40e3f846e99cd245544b9aeebd', 'request_id': 'req-1b84e53e-7f7a-4f47-972f-c191023c0dd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.901697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] The token cache is being invalidate because project f2ffdb40e3f846e99cd245544b9aeebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:46.901923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.902681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1b84e53e-7f7a-4f47-972f-c191023c0dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:46.904589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1172/2347] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:46 2022] DELETE /identity/v3/projects/f2ffdb40e3f846e99cd245544b9aeebd => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:46.909994 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501928-5b81-495b-aac2-449f8601800a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:46.910324 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501928-5b81-495b-aac2-449f8601800a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:46.910545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501928-5b81-495b-aac2-449f8601800a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:46.925627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0dcd6-e177-473e-8503-4b0ec682a0c9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:46.933318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-09501928-5b81-495b-aac2-449f8601800a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:46.934532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-09501928-5b81-495b-aac2-449f8601800a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:46.950900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:46.996553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:46.997420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:46.997835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:46.998234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.000686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:47.020605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1173/2348] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:47.028837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2eea8-ee26-47ef-9001-72b0ee419b4f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.029119 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2eea8-ee26-47ef-9001-72b0ee419b4f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.029412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2eea8-ee26-47ef-9001-72b0ee419b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.049744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-78d2eea8-ee26-47ef-9001-72b0ee419b4f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:47.050792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-78d2eea8-ee26-47ef-9001-72b0ee419b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:47.059858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1174/2349] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:47.064433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2eea8-ee26-47ef-9001-72b0ee419b4f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.073891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2eea8-ee26-47ef-9001-72b0ee419b4f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.078522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.079131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.079436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.079708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.080346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:47.080645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.082258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.083543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.085666 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.094152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1175/2350] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:20:47.099698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1176/2351] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:47.099997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.111389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c07e-95e1-43e9-8bee-01b7a3c46c8e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.118512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70160319-9255-4a52-bd1a-2563c10b16e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.118862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70160319-9255-4a52-bd1a-2563c10b16e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.119141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70160319-9255-4a52-bd1a-2563c10b16e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.119453 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70160319-9255-4a52-bd1a-2563c10b16e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.120204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70160319-9255-4a52-bd1a-2563c10b16e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.122144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70160319-9255-4a52-bd1a-2563c10b16e0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.159062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1176/2352] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:47 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:47.160334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.171923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5960d82-b466-4b48-89a7-cfbe0a80687a service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.232704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:47.244924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ba279-e0c9-42ae-a921-d57080de8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.245544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ba279-e0c9-42ae-a921-d57080de8b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.245827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ba279-e0c9-42ae-a921-d57080de8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.246095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ba279-e0c9-42ae-a921-d57080de8b05 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.246750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ba279-e0c9-42ae-a921-d57080de8b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.248817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ba279-e0c9-42ae-a921-d57080de8b05 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.282119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1177/2353] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:47 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:47.286146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70160319-9255-4a52-bd1a-2563c10b16e0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.297053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70160319-9255-4a52-bd1a-2563c10b16e0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.302270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2856c3-5b8c-4e55-82c8-0df21840ed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.302980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2856c3-5b8c-4e55-82c8-0df21840ed90 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.303323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2856c3-5b8c-4e55-82c8-0df21840ed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.303652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2856c3-5b8c-4e55-82c8-0df21840ed90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.304316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2856c3-5b8c-4e55-82c8-0df21840ed90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.305965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2856c3-5b8c-4e55-82c8-0df21840ed90 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.317498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1177/2354] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:47.321694 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ba279-e0c9-42ae-a921-d57080de8b05 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.334364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ba279-e0c9-42ae-a921-d57080de8b05 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.340356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.341087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.341473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.341801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.342526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.344356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.360029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1178/2355] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:20:47.361646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2856c3-5b8c-4e55-82c8-0df21840ed90 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.377181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2856c3-5b8c-4e55-82c8-0df21840ed90 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.383758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79baec-d3fe-4e33-b813-819f9de4ed53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.384520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79baec-d3fe-4e33-b813-819f9de4ed53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.384846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79baec-d3fe-4e33-b813-819f9de4ed53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.385212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79baec-d3fe-4e33-b813-819f9de4ed53 admin admin] PATH_INFO: `/v3/projects/90897266779e40028dc2be9301efff53/users/7d60bea32ca4476d84d00d9ff95f3516/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.407972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79baec-d3fe-4e33-b813-819f9de4ed53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90897266779e40028dc2be9301efff53, user_id=7d60bea32ca4476d84d00d9ff95f3516, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.410707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79baec-d3fe-4e33-b813-819f9de4ed53 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.428309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1178/2356] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:47 2022] PUT /identity/v3/projects/90897266779e40028dc2be9301efff53/users/7d60bea32ca4476d84d00d9ff95f3516/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:47.431321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.435104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:47.438596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388fecb-2571-4ff7-8edf-df61030ea7c7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.438972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388fecb-2571-4ff7-8edf-df61030ea7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.439267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388fecb-2571-4ff7-8edf-df61030ea7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.452946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33836e0d-69c2-41f5-a41d-6aa109d3a863 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.496389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e388fecb-2571-4ff7-8edf-df61030ea7c7 None None] MFA Rules not processed for user `7d60bea32ca4476d84d00d9ff95f3516`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:47.501622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e388fecb-2571-4ff7-8edf-df61030ea7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:47.589828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.590718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.591788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.592281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] PATH_INFO: `/v3/projects/94736040b12f4702bdc8426f7ec6d4eb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.594102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] RBAC: Authorizing `identity:delete_project(project_id=94736040b12f4702bdc8426f7ec6d4eb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.597521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.630129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1179/2357] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:47.666811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388fecb-2571-4ff7-8edf-df61030ea7c7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.668491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e388fecb-2571-4ff7-8edf-df61030ea7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:47.693075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94736040b12f4702bdc8426f7ec6d4eb', 'request_id': 'req-488b1b3f-0108-43c1-9840-5399dbf2684d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.714724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94736040b12f4702bdc8426f7ec6d4eb', 'request_id': 'req-488b1b3f-0108-43c1-9840-5399dbf2684d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.727146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94736040b12f4702bdc8426f7ec6d4eb', 'request_id': 'req-488b1b3f-0108-43c1-9840-5399dbf2684d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.731480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:47.742897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] The token cache is being invalidate because project 94736040b12f4702bdc8426f7ec6d4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:47.743594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.744605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.756661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1179/2358] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:47 2022] DELETE /identity/v3/projects/94736040b12f4702bdc8426f7ec6d4eb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:47.760822 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:47.762389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:47.770626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388fecb-2571-4ff7-8edf-df61030ea7c7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.850708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.851803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.851803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.851803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.854404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:47.859370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b1b3f-0108-43c1-9840-5399dbf2684d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:47.864733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:47.865438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:47.865724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:47.866003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] PATH_INFO: `/v3/users/de86781742b64c518f8b5e764fc0028b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:47.867166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de86781742b64c518f8b5e764fc0028b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.869396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.872407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:47.958485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de86781742b64c518f8b5e764fc0028b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.959562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de86781742b64c518f8b5e764fc0028b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.969012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de86781742b64c518f8b5e764fc0028b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.969941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de86781742b64c518f8b5e764fc0028b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:47.971009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:47.972518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:47.983210 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1180/2359] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:20:47.987017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de86781742b64c518f8b5e764fc0028b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.004192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1180/2360] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:47 2022] DELETE /identity/v3/users/de86781742b64c518f8b5e764fc0028b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:20:48.448020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:48.462958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d22b42-e76f-4b6a-b736-77e79a179bdf service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:48.529764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:48.530463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:48.530777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:48.531157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] PATH_INFO: `/v3/projects/b802dd46cc0440bc8426bc0caeb3b943` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:48.532697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b802dd46cc0440bc8426bc0caeb3b943)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:48.534716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:48.590005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:48.595249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b802dd46cc0440bc8426bc0caeb3b943', 'request_id': 'req-ffe599dc-a161-4b96-9b71-71fc61e13b9d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.611496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b802dd46cc0440bc8426bc0caeb3b943', 'request_id': 'req-ffe599dc-a161-4b96-9b71-71fc61e13b9d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.611496 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b802dd46cc0440bc8426bc0caeb3b943', 'request_id': 'req-ffe599dc-a161-4b96-9b71-71fc61e13b9d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.624499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] The token cache is being invalidate because project b802dd46cc0440bc8426bc0caeb3b943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:48.624791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.625428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.627062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1181/2361] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:48 2022] DELETE /identity/v3/projects/b802dd46cc0440bc8426bc0caeb3b943 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:48.631022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:48.632474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:48.646566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e28c4-669e-48c9-8d9a-7a10bb6e63c8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:48.709995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:48.710633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:48.710913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:48.711212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] PATH_INFO: `/v3/users/379fc12ec534455fb76aa03d0b552df7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:48.712598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=379fc12ec534455fb76aa03d0b552df7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:48.715654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:48.765098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379fc12ec534455fb76aa03d0b552df7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.765994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379fc12ec534455fb76aa03d0b552df7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.775178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379fc12ec534455fb76aa03d0b552df7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.775839 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379fc12ec534455fb76aa03d0b552df7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.791776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-03d946e9-7064-471c-b8d3-729278276eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379fc12ec534455fb76aa03d0b552df7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:48.807424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:48.813401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1181/2362] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:20:48 2022] DELETE /identity/v3/users/379fc12ec534455fb76aa03d0b552df7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:49.131620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:49.133395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:49.148272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe599dc-a161-4b96-9b71-71fc61e13b9d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:49.222234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:49.222937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.223217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.223693 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] PATH_INFO: `/v3/projects/e40ff3dac18c4abf8b9d7671c3757cff` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.224906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40ff3dac18c4abf8b9d7671c3757cff)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:49.226876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:49.304016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40ff3dac18c4abf8b9d7671c3757cff', 'request_id': 'req-ff545723-caa2-4089-a043-30f40a38adad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:49.316148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40ff3dac18c4abf8b9d7671c3757cff', 'request_id': 'req-ff545723-caa2-4089-a043-30f40a38adad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:49.327092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40ff3dac18c4abf8b9d7671c3757cff', 'request_id': 'req-ff545723-caa2-4089-a043-30f40a38adad'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:49.350059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] The token cache is being invalidate because project e40ff3dac18c4abf8b9d7671c3757cff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:20:49.350349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:49.351042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff545723-caa2-4089-a043-30f40a38adad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:20:49.352659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1182/2363] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:20:49 2022] DELETE /identity/v3/projects/e40ff3dac18c4abf8b9d7671c3757cff => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:49.359691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7507e-8812-4b8b-9b64-00585162a313 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.360065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7507e-8812-4b8b-9b64-00585162a313 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.360368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7507e-8812-4b8b-9b64-00585162a313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.383387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f9c7507e-8812-4b8b-9b64-00585162a313 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:49.384784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f9c7507e-8812-4b8b-9b64-00585162a313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:49.468224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1182/2364] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:49.478219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6128da3-29ad-41c2-b7b8-7c95b072070c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.478789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6128da3-29ad-41c2-b7b8-7c95b072070c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.478918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6128da3-29ad-41c2-b7b8-7c95b072070c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.499200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e6128da3-29ad-41c2-b7b8-7c95b072070c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:49.500451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e6128da3-29ad-41c2-b7b8-7c95b072070c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:49.509689 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1183/2365] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:49.514314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7507e-8812-4b8b-9b64-00585162a313 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:49.526037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7507e-8812-4b8b-9b64-00585162a313 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:49.529179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:49.529796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.530070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.530338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.530992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:20:49.531302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:49.532944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:49.545770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1183/2366] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:20:49.550611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6128da3-29ad-41c2-b7b8-7c95b072070c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:49.562424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6128da3-29ad-41c2-b7b8-7c95b072070c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:49.568070 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac7081-98be-4dae-ab3d-b7701cf7e451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:49.568738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac7081-98be-4dae-ab3d-b7701cf7e451 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.569017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac7081-98be-4dae-ab3d-b7701cf7e451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.569290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac7081-98be-4dae-ab3d-b7701cf7e451 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.570004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac7081-98be-4dae-ab3d-b7701cf7e451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:49.572129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac7081-98be-4dae-ab3d-b7701cf7e451 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:49.603735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1184/2367] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:20:49 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:20:49.608860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:49.621622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1db3b1-dad2-4a8f-bf4a-070d49eda868 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:49.687836 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140e60e-a367-4f64-9db3-116a63710b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:49.688374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140e60e-a367-4f64-9db3-116a63710b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.688652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140e60e-a367-4f64-9db3-116a63710b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.688920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140e60e-a367-4f64-9db3-116a63710b31 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.689554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140e60e-a367-4f64-9db3-116a63710b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:49.691541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140e60e-a367-4f64-9db3-116a63710b31 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:49.715908 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1184/2368] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:20:49 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:20:49.719991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac7081-98be-4dae-ab3d-b7701cf7e451 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:49.734846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac7081-98be-4dae-ab3d-b7701cf7e451 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:49.742616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850806a-8357-4f2b-a2b7-5521040ba1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:49.743205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850806a-8357-4f2b-a2b7-5521040ba1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.743586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850806a-8357-4f2b-a2b7-5521040ba1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.743924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850806a-8357-4f2b-a2b7-5521040ba1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.744565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850806a-8357-4f2b-a2b7-5521040ba1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:49.746327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850806a-8357-4f2b-a2b7-5521040ba1b9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:49.757384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1185/2369] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:49.761808 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140e60e-a367-4f64-9db3-116a63710b31 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:49.772043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140e60e-a367-4f64-9db3-116a63710b31 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:49.777308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e724cf0-33a7-4292-a885-00c97617c237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:49.777918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724cf0-33a7-4292-a885-00c97617c237 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.778192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724cf0-33a7-4292-a885-00c97617c237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.778470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724cf0-33a7-4292-a885-00c97617c237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.779102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e724cf0-33a7-4292-a885-00c97617c237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:49.780718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e724cf0-33a7-4292-a885-00c97617c237 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:49.793832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1185/2370] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:20:49.796579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850806a-8357-4f2b-a2b7-5521040ba1b9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:49.808611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850806a-8357-4f2b-a2b7-5521040ba1b9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:49.814293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:49.814959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.815253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.815542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] PATH_INFO: `/v3/projects/a34eb3e0aba44efb8ddafedd89df4060/users/37a3870e9df04d7795a649a21b722801/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.832681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34eb3e0aba44efb8ddafedd89df4060, user_id=37a3870e9df04d7795a649a21b722801, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:49.835565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:49.845341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:49.852564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1186/2371] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:20:49 2022] PUT /identity/v3/projects/a34eb3e0aba44efb8ddafedd89df4060/users/37a3870e9df04d7795a649a21b722801/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:20:49.858285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2fb57-db26-4df6-9098-2b86e175d464 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:49.858594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2fb57-db26-4df6-9098-2b86e175d464 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:49.858866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2fb57-db26-4df6-9098-2b86e175d464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:49.878183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0ba2fb57-db26-4df6-9098-2b86e175d464 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:49.879417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0ba2fb57-db26-4df6-9098-2b86e175d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:49.944842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1186/2372] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:49.956983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:49.961107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:50.032959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd4e64-620b-411a-9b20-19a2fb82a605 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:50.036984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:50.037596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:50.037869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:50.038134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:50.054396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:50.055908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:50.066366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1187/2373] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:50.937302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e75a6-b227-432c-8a3d-968eb9dcf134 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:50.937849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e75a6-b227-432c-8a3d-968eb9dcf134 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:50.937935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e75a6-b227-432c-8a3d-968eb9dcf134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:50.947350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:20:50.971446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-305e75a6-b227-432c-8a3d-968eb9dcf134 None None] MFA Rules not processed for user `37a3870e9df04d7795a649a21b722801`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:50.972987 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-305e75a6-b227-432c-8a3d-968eb9dcf134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:51.062991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1187/2374] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:20:51.077875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:51.079588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:51.157470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd116d4-fcfc-4078-a5c4-a42bf5f7a8c4 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:51.161751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686158a9-4baf-47bb-b9d1-24d558d8014b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:51.162518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686158a9-4baf-47bb-b9d1-24d558d8014b service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:51.162749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686158a9-4baf-47bb-b9d1-24d558d8014b service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:51.163020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686158a9-4baf-47bb-b9d1-24d558d8014b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:51.173756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686158a9-4baf-47bb-b9d1-24d558d8014b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:51.175282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686158a9-4baf-47bb-b9d1-24d558d8014b service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:51.186412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1188/2375] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:57.699074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e75a6-b227-432c-8a3d-968eb9dcf134 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:20:57.701504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-305e75a6-b227-432c-8a3d-968eb9dcf134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:57.715004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e75a6-b227-432c-8a3d-968eb9dcf134 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:20:57.720037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72f3a7-2616-459c-b627-3e8cff11afe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:20:57.720653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72f3a7-2616-459c-b627-3e8cff11afe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:57.721152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72f3a7-2616-459c-b627-3e8cff11afe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:57.721656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72f3a7-2616-459c-b627-3e8cff11afe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:57.724985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-db72f3a7-2616-459c-b627-3e8cff11afe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:57.804318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72f3a7-2616-459c-b627-3e8cff11afe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:20:57.805952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72f3a7-2616-459c-b627-3e8cff11afe0 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:20:57.817092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1188/2376] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:20:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:20:57.903460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6bf20-6cd7-4553-88c1-a3cfcac28c7b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:57.904565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6bf20-6cd7-4553-88c1-a3cfcac28c7b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:57.905030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6bf20-6cd7-4553-88c1-a3cfcac28c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:57.937759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cae6bf20-6cd7-4553-88c1-a3cfcac28c7b None None] MFA Rules not processed for user `7d60bea32ca4476d84d00d9ff95f3516`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:57.938992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cae6bf20-6cd7-4553-88c1-a3cfcac28c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:57.952476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fde50-916c-4b36-9271-ecb0cae4cdf5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:20:57.952697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fde50-916c-4b36-9271-ecb0cae4cdf5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:20:57.952913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fde50-916c-4b36-9271-ecb0cae4cdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:20:57.976583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-868fde50-916c-4b36-9271-ecb0cae4cdf5 None None] MFA Rules not processed for user `37a3870e9df04d7795a649a21b722801`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:20:57.977997 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-868fde50-916c-4b36-9271-ecb0cae4cdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:20:57.986661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1189/2377] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:20:58.018845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1189/2378] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:01.412199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fde50-916c-4b36-9271-ecb0cae4cdf5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:01.416215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-868fde50-916c-4b36-9271-ecb0cae4cdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:01.460105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fde50-916c-4b36-9271-ecb0cae4cdf5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:01.486686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:01.491662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:01.491964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:01.495775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] PATH_INFO: `/v3/users/e736dfaf2c704ef2986f9880503876be` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:01.498471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e736dfaf2c704ef2986f9880503876be)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:01.505551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:01.600122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e736dfaf2c704ef2986f9880503876be'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:01.601320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e736dfaf2c704ef2986f9880503876be'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:01.614747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e736dfaf2c704ef2986f9880503876be'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:01.615772 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e736dfaf2c704ef2986f9880503876be'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:01.635633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e736dfaf2c704ef2986f9880503876be'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:01.641940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:01.664818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1190/2379] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:21:01 2022] DELETE /identity/v3/users/e736dfaf2c704ef2986f9880503876be => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:01.669870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e743e-ccd3-4287-8d45-b0c634dffe81 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:01.670907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e743e-ccd3-4287-8d45-b0c634dffe81 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:01.670907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e743e-ccd3-4287-8d45-b0c634dffe81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:01.693115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3d1e743e-ccd3-4287-8d45-b0c634dffe81 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:01.694341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3d1e743e-ccd3-4287-8d45-b0c634dffe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:01.762207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:01.797093 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1190/2380] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:21:01.827132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:01.829328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:02.013969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90d4f3-53ae-4b64-96e0-1325e64cdef4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:02.021583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e181b-a9fe-4817-8573-c7a1973c11f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:02.021583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e181b-a9fe-4817-8573-c7a1973c11f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:02.021583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e181b-a9fe-4817-8573-c7a1973c11f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:02.021583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e181b-a9fe-4817-8573-c7a1973c11f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:02.038092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e181b-a9fe-4817-8573-c7a1973c11f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:02.043158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e181b-a9fe-4817-8573-c7a1973c11f5 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:02.065923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1191/2381] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:02.600180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e743e-ccd3-4287-8d45-b0c634dffe81 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:02.603163 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3d1e743e-ccd3-4287-8d45-b0c634dffe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:02.671787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e743e-ccd3-4287-8d45-b0c634dffe81 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:02.678960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:02.680153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:02.680719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:02.681429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] PATH_INFO: `/v3/projects/570d114f4722462e8dc8d51be8b69c3d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:02.683117 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570d114f4722462e8dc8d51be8b69c3d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:02.686197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:02.797006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570d114f4722462e8dc8d51be8b69c3d', 'request_id': 'req-7b6fdb08-1530-4aaa-8b58-36184af69478'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:02.808066 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570d114f4722462e8dc8d51be8b69c3d', 'request_id': 'req-7b6fdb08-1530-4aaa-8b58-36184af69478'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:02.817788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570d114f4722462e8dc8d51be8b69c3d', 'request_id': 'req-7b6fdb08-1530-4aaa-8b58-36184af69478'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:02.842913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] The token cache is being invalidate because project 570d114f4722462e8dc8d51be8b69c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:21:02.843206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:02.844059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b6fdb08-1530-4aaa-8b58-36184af69478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:02.845528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1191/2382] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:21:02 2022] DELETE /identity/v3/projects/570d114f4722462e8dc8d51be8b69c3d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:02.850923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648332d5-59e9-4571-a024-ca5b7bcdf31a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:02.851597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648332d5-59e9-4571-a024-ca5b7bcdf31a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:02.851745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648332d5-59e9-4571-a024-ca5b7bcdf31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:02.886030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-648332d5-59e9-4571-a024-ca5b7bcdf31a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:02.887269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-648332d5-59e9-4571-a024-ca5b7bcdf31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:02.980190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1192/2383] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:21:02.988811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88a10d-99e1-4ead-827f-a3951da82ad3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:02.989353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88a10d-99e1-4ead-827f-a3951da82ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:02.989633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88a10d-99e1-4ead-827f-a3951da82ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:03.018386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0a88a10d-99e1-4ead-827f-a3951da82ad3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:03.020383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0a88a10d-99e1-4ead-827f-a3951da82ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:03.032525 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1192/2384] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:03.034034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648332d5-59e9-4571-a024-ca5b7bcdf31a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:03.047853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648332d5-59e9-4571-a024-ca5b7bcdf31a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:03.053265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:03.053899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:03.054284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:03.054737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:03.055764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:21:03.056081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:03.058080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:03.076648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1193/2385] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:21:03.080622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88a10d-99e1-4ead-827f-a3951da82ad3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:03.098269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88a10d-99e1-4ead-827f-a3951da82ad3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:03.107747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce67c0-3fb1-4fc7-8eca-d3f7aef7a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:03.108860 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce67c0-3fb1-4fc7-8eca-d3f7aef7a8ba admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:03.109214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce67c0-3fb1-4fc7-8eca-d3f7aef7a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:03.109542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce67c0-3fb1-4fc7-8eca-d3f7aef7a8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:03.110282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce67c0-3fb1-4fc7-8eca-d3f7aef7a8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:03.112366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce67c0-3fb1-4fc7-8eca-d3f7aef7a8ba admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:03.162601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1193/2386] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:21:03 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:21:03.168454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:03.192115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19d598-cf33-47bb-943f-aea0c6209915 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:03.296328 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa06785-dc80-4670-a37d-8d84a2b1b031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:03.297339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa06785-dc80-4670-a37d-8d84a2b1b031 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:03.298266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa06785-dc80-4670-a37d-8d84a2b1b031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:03.298704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa06785-dc80-4670-a37d-8d84a2b1b031 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:03.299548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa06785-dc80-4670-a37d-8d84a2b1b031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:03.304018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa06785-dc80-4670-a37d-8d84a2b1b031 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:03.336306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1194/2387] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:21:03 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:21:03.342330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce67c0-3fb1-4fc7-8eca-d3f7aef7a8ba admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:03.362563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce67c0-3fb1-4fc7-8eca-d3f7aef7a8ba admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:03.373407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229554c-5e20-453c-acc5-1d5768f7e71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:03.374510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229554c-5e20-453c-acc5-1d5768f7e71a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:03.375224 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229554c-5e20-453c-acc5-1d5768f7e71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:03.375500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229554c-5e20-453c-acc5-1d5768f7e71a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:03.376703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229554c-5e20-453c-acc5-1d5768f7e71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:03.379901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229554c-5e20-453c-acc5-1d5768f7e71a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:03.404411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1194/2388] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:21:03.410781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa06785-dc80-4670-a37d-8d84a2b1b031 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:03.433482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa06785-dc80-4670-a37d-8d84a2b1b031 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:03.444668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd47bd-5a24-4e32-b194-72ec9ad373da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:03.447093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd47bd-5a24-4e32-b194-72ec9ad373da admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:03.448443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd47bd-5a24-4e32-b194-72ec9ad373da admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:03.448443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd47bd-5a24-4e32-b194-72ec9ad373da admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:03.450093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd47bd-5a24-4e32-b194-72ec9ad373da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:03.453982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd47bd-5a24-4e32-b194-72ec9ad373da admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:03.488853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229554c-5e20-453c-acc5-1d5768f7e71a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:03.489639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1195/2389] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:21:03.502814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229554c-5e20-453c-acc5-1d5768f7e71a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:03.512782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d173-dc55-4ba5-95de-f8bd00e5a2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:03.513542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d173-dc55-4ba5-95de-f8bd00e5a2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:03.513924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d173-dc55-4ba5-95de-f8bd00e5a2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:03.514300 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d173-dc55-4ba5-95de-f8bd00e5a2d9 admin admin] PATH_INFO: `/v3/projects/6bad46d0566c4d869aacb61597d6d3c5/users/c084c7d120a049b1bd3416cbc36c3301/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:03.545708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35d173-dc55-4ba5-95de-f8bd00e5a2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bad46d0566c4d869aacb61597d6d3c5, user_id=c084c7d120a049b1bd3416cbc36c3301, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:03.555297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35d173-dc55-4ba5-95de-f8bd00e5a2d9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:03.599047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1195/2390] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:21:03 2022] PUT /identity/v3/projects/6bad46d0566c4d869aacb61597d6d3c5/users/c084c7d120a049b1bd3416cbc36c3301/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:03.684299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49167021-b0b2-4f63-ba9c-f732c89c4af1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:03.685715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49167021-b0b2-4f63-ba9c-f732c89c4af1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:03.685715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49167021-b0b2-4f63-ba9c-f732c89c4af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:03.697875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:03.779682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-49167021-b0b2-4f63-ba9c-f732c89c4af1 None None] MFA Rules not processed for user `c084c7d120a049b1bd3416cbc36c3301`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:03.781115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-49167021-b0b2-4f63-ba9c-f732c89c4af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:03.999946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d173-dc55-4ba5-95de-f8bd00e5a2d9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:04.003722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1196/2391] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:21:04.122831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d173-dc55-4ba5-95de-f8bd00e5a2d9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:04.139107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47fec-2641-499e-8d38-dfdc49517e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:04.139948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47fec-2641-499e-8d38-dfdc49517e11 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:04.140328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47fec-2641-499e-8d38-dfdc49517e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:04.140696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47fec-2641-499e-8d38-dfdc49517e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:04.141448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf47fec-2641-499e-8d38-dfdc49517e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:04.145772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf47fec-2641-499e-8d38-dfdc49517e11 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:04.185413 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:04.212470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1196/2392] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:21:03 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:21:04.213829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49167021-b0b2-4f63-ba9c-f732c89c4af1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:04.230552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49167021-b0b2-4f63-ba9c-f732c89c4af1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:04.385871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b539-c269-4217-8107-e0220fcc8ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:04.386999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b539-c269-4217-8107-e0220fcc8ecc admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:04.389848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b539-c269-4217-8107-e0220fcc8ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:04.389848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b539-c269-4217-8107-e0220fcc8ecc admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:04.392017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4b539-c269-4217-8107-e0220fcc8ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:04.395978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4b539-c269-4217-8107-e0220fcc8ecc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:04.441479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1197/2393] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:21:04 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:21:04.443658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47fec-2641-499e-8d38-dfdc49517e11 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:04.463622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47fec-2641-499e-8d38-dfdc49517e11 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:04.471440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d1b05-6aa4-4dc6-9291-e1c56dcf3d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:04.473792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1b05-6aa4-4dc6-9291-e1c56dcf3d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:04.474265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1b05-6aa4-4dc6-9291-e1c56dcf3d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:04.474711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1b05-6aa4-4dc6-9291-e1c56dcf3d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:04.475856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d1b05-6aa4-4dc6-9291-e1c56dcf3d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:04.478714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d1b05-6aa4-4dc6-9291-e1c56dcf3d93 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:04.499321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1197/2394] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:21:04.504218 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b539-c269-4217-8107-e0220fcc8ecc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:04.520539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b539-c269-4217-8107-e0220fcc8ecc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:04.531459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e53db-f5d9-40ff-ac40-5c303be0218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:04.532095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e53db-f5d9-40ff-ac40-5c303be0218f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:04.532369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e53db-f5d9-40ff-ac40-5c303be0218f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:04.532632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e53db-f5d9-40ff-ac40-5c303be0218f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:04.533283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e53db-f5d9-40ff-ac40-5c303be0218f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:04.535318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e53db-f5d9-40ff-ac40-5c303be0218f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:04.564063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1198/2395] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:21:04.566927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d1b05-6aa4-4dc6-9291-e1c56dcf3d93 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:04.585385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d1b05-6aa4-4dc6-9291-e1c56dcf3d93 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:04.593732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbb7a2-056f-4f9c-8b30-2eca359eb8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:04.594710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbb7a2-056f-4f9c-8b30-2eca359eb8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:04.595123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbb7a2-056f-4f9c-8b30-2eca359eb8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:04.595557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbb7a2-056f-4f9c-8b30-2eca359eb8b2 admin admin] PATH_INFO: `/v3/projects/820ca8d4ccac4c0399793941f3295215/users/d98620855d534996adcf39ef38c00f4e/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:04.634884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbb7a2-056f-4f9c-8b30-2eca359eb8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820ca8d4ccac4c0399793941f3295215, user_id=d98620855d534996adcf39ef38c00f4e, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:04.643834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbb7a2-056f-4f9c-8b30-2eca359eb8b2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:04.683101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1198/2396] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:21:04 2022] PUT /identity/v3/projects/820ca8d4ccac4c0399793941f3295215/users/d98620855d534996adcf39ef38c00f4e/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:21:04.698408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d8564-0a81-4cfe-a125-6d3a1a1a99e6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:04.698739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d8564-0a81-4cfe-a125-6d3a1a1a99e6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:04.699003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d8564-0a81-4cfe-a125-6d3a1a1a99e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:04.751399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9c2d8564-0a81-4cfe-a125-6d3a1a1a99e6 None None] MFA Rules not processed for user `d98620855d534996adcf39ef38c00f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:04.752817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9c2d8564-0a81-4cfe-a125-6d3a1a1a99e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:04.912571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:04.923409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1199/2397] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:04.925297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbb7a2-056f-4f9c-8b30-2eca359eb8b2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:05.065370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbb7a2-056f-4f9c-8b30-2eca359eb8b2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:05.075110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c2d9d-3198-40ae-b69a-13e41327be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:05.075916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c2d9d-3198-40ae-b69a-13e41327be77 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:05.076425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c2d9d-3198-40ae-b69a-13e41327be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:05.076704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c2d9d-3198-40ae-b69a-13e41327be77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:05.078080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c2d9d-3198-40ae-b69a-13e41327be77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:05.079802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c2d9d-3198-40ae-b69a-13e41327be77 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:05.129698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1199/2398] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:21:04 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:21:05.134235 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d8564-0a81-4cfe-a125-6d3a1a1a99e6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:05.149465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d8564-0a81-4cfe-a125-6d3a1a1a99e6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:05.254402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4df69-ebb7-424b-8f6a-bbd03d5682df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:05.254402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4df69-ebb7-424b-8f6a-bbd03d5682df admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:05.254402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4df69-ebb7-424b-8f6a-bbd03d5682df admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:05.254981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4df69-ebb7-424b-8f6a-bbd03d5682df admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:05.260522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4df69-ebb7-424b-8f6a-bbd03d5682df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:05.260522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4df69-ebb7-424b-8f6a-bbd03d5682df admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:05.291116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1200/2399] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:21:05 2022] POST /identity/v3/users => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:21:05.292142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c2d9d-3198-40ae-b69a-13e41327be77 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:05.313121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c2d9d-3198-40ae-b69a-13e41327be77 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:05.336241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d3722-7de0-497b-baf0-5ab8a540548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:05.337360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d3722-7de0-497b-baf0-5ab8a540548b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:05.337641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d3722-7de0-497b-baf0-5ab8a540548b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:05.341498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d3722-7de0-497b-baf0-5ab8a540548b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:05.342339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d3722-7de0-497b-baf0-5ab8a540548b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:05.344197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d3722-7de0-497b-baf0-5ab8a540548b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:05.392474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1200/2400] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:21:05.398255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4df69-ebb7-424b-8f6a-bbd03d5682df admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:05.428134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4df69-ebb7-424b-8f6a-bbd03d5682df admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:05.439866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b0fbe-d898-4675-a2fd-0b87c018634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:05.440562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b0fbe-d898-4675-a2fd-0b87c018634c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:05.440846 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b0fbe-d898-4675-a2fd-0b87c018634c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:05.441177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b0fbe-d898-4675-a2fd-0b87c018634c admin admin] PATH_INFO: `/v3/projects/e3a0ae61abd84f7f844021056f3ce091/users/19d16e9bf4c2478e97fcc279cf3b24fc/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:05.468704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b0fbe-d898-4675-a2fd-0b87c018634c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a0ae61abd84f7f844021056f3ce091, user_id=19d16e9bf4c2478e97fcc279cf3b24fc, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:05.474078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b0fbe-d898-4675-a2fd-0b87c018634c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:05.516741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1201/2401] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:21:05 2022] PUT /identity/v3/projects/e3a0ae61abd84f7f844021056f3ce091/users/19d16e9bf4c2478e97fcc279cf3b24fc/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:05.521265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d3722-7de0-497b-baf0-5ab8a540548b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:05.532558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d3722-7de0-497b-baf0-5ab8a540548b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:05.615439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969b90d-d097-493d-836e-743b17efc566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:05.616000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969b90d-d097-493d-836e-743b17efc566 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:05.616330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969b90d-d097-493d-836e-743b17efc566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:05.616991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969b90d-d097-493d-836e-743b17efc566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:05.617417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969b90d-d097-493d-836e-743b17efc566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:05.619626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969b90d-d097-493d-836e-743b17efc566 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:05.635593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1201/2402] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:21:05.640076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b0fbe-d898-4675-a2fd-0b87c018634c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:05.655289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b0fbe-d898-4675-a2fd-0b87c018634c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:05.662804 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:05.664989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:05.664989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:05.665435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] PATH_INFO: `/v3/projects/e3a0ae61abd84f7f844021056f3ce091/users/19d16e9bf4c2478e97fcc279cf3b24fc/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:05.668287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a0ae61abd84f7f844021056f3ce091, user_id=19d16e9bf4c2478e97fcc279cf3b24fc, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:05.672823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:05.685205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1202/2403] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:21:05 2022] PUT /identity/v3/projects/e3a0ae61abd84f7f844021056f3ce091/users/19d16e9bf4c2478e97fcc279cf3b24fc/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:05.697285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9ac2-e794-4a40-b7f3-dbb66646fcef None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:05.697603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9ac2-e794-4a40-b7f3-dbb66646fcef None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:05.697835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9ac2-e794-4a40-b7f3-dbb66646fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:05.746614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-02cb9ac2-e794-4a40-b7f3-dbb66646fcef None None] MFA Rules not processed for user `19d16e9bf4c2478e97fcc279cf3b24fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:05.747952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-02cb9ac2-e794-4a40-b7f3-dbb66646fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:05.771759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:05.893961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:05.896803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:05.907279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1202/2404] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:05.999094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83225bec-5428-4c80-9454-e1536d2babfb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:06.018649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeedc77-c646-4154-acfe-3e7a171cc5c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:06.019948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeedc77-c646-4154-acfe-3e7a171cc5c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:06.020273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeedc77-c646-4154-acfe-3e7a171cc5c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:06.020817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeedc77-c646-4154-acfe-3e7a171cc5c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:06.106780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeedc77-c646-4154-acfe-3e7a171cc5c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:06.109220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeedc77-c646-4154-acfe-3e7a171cc5c6 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:06.131707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1203/2405] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:21:07.690110 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9ac2-e794-4a40-b7f3-dbb66646fcef None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:07.692799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-02cb9ac2-e794-4a40-b7f3-dbb66646fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:07.814238 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9ac2-e794-4a40-b7f3-dbb66646fcef None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:07.820971 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:07.821917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:07.822435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:07.822921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] PATH_INFO: `/v3/users/045a25aae6164763860a0227f5ea5bce` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:07.824761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=045a25aae6164763860a0227f5ea5bce)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:07.827192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:07.951995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '045a25aae6164763860a0227f5ea5bce'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:07.951995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '045a25aae6164763860a0227f5ea5bce'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:07.971920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '045a25aae6164763860a0227f5ea5bce'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:07.975978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeedc77-c646-4154-acfe-3e7a171cc5c6 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:07.983969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '045a25aae6164763860a0227f5ea5bce'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:07.998461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bf55f9bc-3bbf-4564-8a20-dbfc22c503a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '045a25aae6164763860a0227f5ea5bce'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:08.000943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeedc77-c646-4154-acfe-3e7a171cc5c6 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:08.008254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:08.008682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1203/2406] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:21:07 2022] DELETE /identity/v3/users/045a25aae6164763860a0227f5ea5bce => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:08.015173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133118f-ec01-462f-b34f-3191fa7af99a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:08.015489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133118f-ec01-462f-b34f-3191fa7af99a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:08.015763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133118f-ec01-462f-b34f-3191fa7af99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:08.020138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:08.020589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:08.021047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:08.023223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:08.037801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:08.041621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6133118f-ec01-462f-b34f-3191fa7af99a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:08.043370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6133118f-ec01-462f-b34f-3191fa7af99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:08.099323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:08.155448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:08.158483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:08.159891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1204/2407] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:08.173302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1204/2408] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:08.186193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133118f-ec01-462f-b34f-3191fa7af99a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:08.189382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6133118f-ec01-462f-b34f-3191fa7af99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:08.382651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133118f-ec01-462f-b34f-3191fa7af99a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:08.396528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:08.397510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:08.397727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:08.398271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:08.414122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:08.418283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:08.439189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1205/2409] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:09.075626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:09.077095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:09.088935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a52ded-fba1-471e-bfff-8f15e1afdc8f service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:09.165102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1b1f0-d1a2-42f5-b56f-1c4ec37c66b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:09.165967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b1f0-d1a2-42f5-b56f-1c4ec37c66b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:09.167413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b1f0-d1a2-42f5-b56f-1c4ec37c66b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:09.167854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b1f0-d1a2-42f5-b56f-1c4ec37c66b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:09.170485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7ae1b1f0-d1a2-42f5-b56f-1c4ec37c66b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:09.343924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1b1f0-d1a2-42f5-b56f-1c4ec37c66b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:09.351046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1b1f0-d1a2-42f5-b56f-1c4ec37c66b7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:09.413622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1205/2410] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:09.740257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:09.741707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:09.776341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85ee98-0508-4ff9-a1fe-51de933b4821 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:09.797762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:09.798429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:09.798704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:09.798981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] PATH_INFO: `/v3/projects/7d5986426ceb4864bd44f0f60b95933f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:09.806125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5986426ceb4864bd44f0f60b95933f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:09.808319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:09.958743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5986426ceb4864bd44f0f60b95933f', 'request_id': 'req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:09.972807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5986426ceb4864bd44f0f60b95933f', 'request_id': 'req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:09.984716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5986426ceb4864bd44f0f60b95933f', 'request_id': 'req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:10.008350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] The token cache is being invalidate because project 7d5986426ceb4864bd44f0f60b95933f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:21:10.008796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:10.010166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-747c7e80-b3fe-42e9-bd70-8cc4aa6aebd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:10.015779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1206/2411] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:21:09 2022] DELETE /identity/v3/projects/7d5986426ceb4864bd44f0f60b95933f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:11.749013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db0d3-327c-4c17-8728-8bffc46e9af3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:11.749577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db0d3-327c-4c17-8728-8bffc46e9af3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:11.749577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db0d3-327c-4c17-8728-8bffc46e9af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:11.790708 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e72db0d3-327c-4c17-8728-8bffc46e9af3 None None] MFA Rules not processed for user `4c3e5c1776b8470a9b677bac8dfbc9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:11.792299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e72db0d3-327c-4c17-8728-8bffc46e9af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:11.868324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:11.880230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1206/2412] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:13.212972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eace10-8371-455d-a928-20d5312132d1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:13.213902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eace10-8371-455d-a928-20d5312132d1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:13.214381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eace10-8371-455d-a928-20d5312132d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:13.251079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-95eace10-8371-455d-a928-20d5312132d1 None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:13.253363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95eace10-8371-455d-a928-20d5312132d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:13.394340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1207/2413] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 31 12:21:13.401666 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72db0d3-327c-4c17-8728-8bffc46e9af3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:13.403532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e72db0d3-327c-4c17-8728-8bffc46e9af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:13.460190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deeacb-1668-487f-8717-ff239595c01a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:13.460473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deeacb-1668-487f-8717-ff239595c01a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:13.460772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deeacb-1668-487f-8717-ff239595c01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:13.472230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:13.500872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e1deeacb-1668-487f-8717-ff239595c01a None None] MFA Rules not processed for user `a8320be5e9644b6699ac526913e109eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:13.503212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e1deeacb-1668-487f-8717-ff239595c01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:13.554869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72db0d3-327c-4c17-8728-8bffc46e9af3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:13.567319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b27a6-f26e-4dc8-a5e7-57ef165e647a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:13.572147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b27a6-f26e-4dc8-a5e7-57ef165e647a service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:13.572652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b27a6-f26e-4dc8-a5e7-57ef165e647a service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:13.573074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b27a6-f26e-4dc8-a5e7-57ef165e647a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:13.592187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1208/2414] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:13.594867 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b27a6-f26e-4dc8-a5e7-57ef165e647a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:13.611512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b27a6-f26e-4dc8-a5e7-57ef165e647a service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:13.626121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1207/2415] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:21:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:21:14.537860 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1deeacb-1668-487f-8717-ff239595c01a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:14.540193 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e1deeacb-1668-487f-8717-ff239595c01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:14.662975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1deeacb-1668-487f-8717-ff239595c01a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:14.670665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:14.671541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:14.671942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:14.672258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] PATH_INFO: `/v3/users/c084c7d120a049b1bd3416cbc36c3301` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:14.673508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c084c7d120a049b1bd3416cbc36c3301)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:14.675761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:14.781897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c084c7d120a049b1bd3416cbc36c3301'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:14.782568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c084c7d120a049b1bd3416cbc36c3301'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:14.799329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c084c7d120a049b1bd3416cbc36c3301'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:14.847688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c084c7d120a049b1bd3416cbc36c3301'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:14.862237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c084c7d120a049b1bd3416cbc36c3301'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:14.886943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1209/2416] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:21:14 2022] DELETE /identity/v3/users/c084c7d120a049b1bd3416cbc36c3301 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:14.902840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161c2a3-18ec-4e9a-9855-55a700d6a377 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:14.903917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161c2a3-18ec-4e9a-9855-55a700d6a377 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:14.904398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161c2a3-18ec-4e9a-9855-55a700d6a377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:14.971677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c161c2a3-18ec-4e9a-9855-55a700d6a377 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:14.978791 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c161c2a3-18ec-4e9a-9855-55a700d6a377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:15.167508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1208/2417] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:21:15.188232 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:15.190131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:15.263812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:15.317222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4753f-a773-49a3-9572-3622da629f84 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:15.329812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95674f0-2f89-4ba0-b97b-c3167db848d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:15.330457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95674f0-2f89-4ba0-b97b-c3167db848d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:15.330735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95674f0-2f89-4ba0-b97b-c3167db848d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:15.331083 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95674f0-2f89-4ba0-b97b-c3167db848d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:15.344736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95674f0-2f89-4ba0-b97b-c3167db848d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:15.348225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95674f0-2f89-4ba0-b97b-c3167db848d5 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:15.364935 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1210/2418] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:16.030876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161c2a3-18ec-4e9a-9855-55a700d6a377 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:16.032513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c161c2a3-18ec-4e9a-9855-55a700d6a377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:16.061013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161c2a3-18ec-4e9a-9855-55a700d6a377 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:16.076109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:16.077090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:16.077801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:16.078177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] PATH_INFO: `/v3/projects/6bad46d0566c4d869aacb61597d6d3c5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:16.080947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bad46d0566c4d869aacb61597d6d3c5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:16.091306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:16.255061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bad46d0566c4d869aacb61597d6d3c5', 'request_id': 'req-94450e23-5c38-4e88-99ab-8a362be6fac3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.278221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bad46d0566c4d869aacb61597d6d3c5', 'request_id': 'req-94450e23-5c38-4e88-99ab-8a362be6fac3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.290428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bad46d0566c4d869aacb61597d6d3c5', 'request_id': 'req-94450e23-5c38-4e88-99ab-8a362be6fac3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.309599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] The token cache is being invalidate because project 6bad46d0566c4d869aacb61597d6d3c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:21:16.310186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.311219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.313115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1209/2419] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:21:16 2022] DELETE /identity/v3/projects/6bad46d0566c4d869aacb61597d6d3c5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:16.317349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95674f0-2f89-4ba0-b97b-c3167db848d5 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:16.331858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95674f0-2f89-4ba0-b97b-c3167db848d5 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:16.493951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:16.494870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:16.495164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:16.495461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] PATH_INFO: `/v3/users/d98620855d534996adcf39ef38c00f4e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:16.497262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98620855d534996adcf39ef38c00f4e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:16.499347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:16.621038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98620855d534996adcf39ef38c00f4e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.621800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98620855d534996adcf39ef38c00f4e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.671123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98620855d534996adcf39ef38c00f4e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.715840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98620855d534996adcf39ef38c00f4e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.727001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98620855d534996adcf39ef38c00f4e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:16.752818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1211/2420] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:21:16 2022] DELETE /identity/v3/users/d98620855d534996adcf39ef38c00f4e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:17.332623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:17.334654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:17.362390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94450e23-5c38-4e88-99ab-8a362be6fac3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:17.382100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:17.569375 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:17.570135 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:17.570468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:17.570904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] PATH_INFO: `/v3/projects/820ca8d4ccac4c0399793941f3295215` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:17.573074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=820ca8d4ccac4c0399793941f3295215)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:17.575303 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:17.687110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820ca8d4ccac4c0399793941f3295215', 'request_id': 'req-87bb902f-9b50-4a5c-b8bf-d2488bd46417'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:17.704368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820ca8d4ccac4c0399793941f3295215', 'request_id': 'req-87bb902f-9b50-4a5c-b8bf-d2488bd46417'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:17.715935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820ca8d4ccac4c0399793941f3295215', 'request_id': 'req-87bb902f-9b50-4a5c-b8bf-d2488bd46417'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:17.731323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] The token cache is being invalidate because project 820ca8d4ccac4c0399793941f3295215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:21:17.731758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:17.732389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:17.736484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1210/2421] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:21:17 2022] DELETE /identity/v3/projects/820ca8d4ccac4c0399793941f3295215 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:21:17.740501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:17.743194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:17.767853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7411248-7bfe-44af-b195-04406a1616a0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:17.850780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:17.867310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:17.868674 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:17.869124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:17.870107 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] PATH_INFO: `/v3/users/19d16e9bf4c2478e97fcc279cf3b24fc` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:17.875399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d16e9bf4c2478e97fcc279cf3b24fc)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:17.877293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:18.026981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d16e9bf4c2478e97fcc279cf3b24fc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:18.028430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d16e9bf4c2478e97fcc279cf3b24fc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:18.073036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d16e9bf4c2478e97fcc279cf3b24fc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:18.082115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d16e9bf4c2478e97fcc279cf3b24fc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:18.093898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2f97bc71-915d-4be8-b40d-d3b46ef80a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d16e9bf4c2478e97fcc279cf3b24fc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:18.102933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1212/2422] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:21:17 2022] DELETE /identity/v3/users/19d16e9bf4c2478e97fcc279cf3b24fc => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:19.711597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:19.714250 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:19.733943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb902f-9b50-4a5c-b8bf-d2488bd46417 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:19.876596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:19.876596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:19.876596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:19.876596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] PATH_INFO: `/v3/projects/e3a0ae61abd84f7f844021056f3ce091` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:19.877227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a0ae61abd84f7f844021056f3ce091)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:19.881080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:20.042077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:20.046554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a0ae61abd84f7f844021056f3ce091', 'request_id': 'req-880766af-186d-4a07-b32f-59d3c18c3951'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:20.065918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a0ae61abd84f7f844021056f3ce091', 'request_id': 'req-880766af-186d-4a07-b32f-59d3c18c3951'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:20.078649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a0ae61abd84f7f844021056f3ce091', 'request_id': 'req-880766af-186d-4a07-b32f-59d3c18c3951'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:20.112903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] The token cache is being invalidate because project e3a0ae61abd84f7f844021056f3ce091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:21:20.113278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:20.113903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:20.116198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1211/2423] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:21:19 2022] DELETE /identity/v3/projects/e3a0ae61abd84f7f844021056f3ce091 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:28.097846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:30.102865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:32.750482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038cec-d04b-4f91-987c-ae07dfb650cd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:32.751050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038cec-d04b-4f91-987c-ae07dfb650cd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:32.751050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038cec-d04b-4f91-987c-ae07dfb650cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:32.778640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4e038cec-d04b-4f91-987c-ae07dfb650cd None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:32.780003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4e038cec-d04b-4f91-987c-ae07dfb650cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:32.869919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1213/2424] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:21:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:32.884316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:32.886481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:32.988621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880766af-186d-4a07-b32f-59d3c18c3951 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:32.996127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fb4b-1138-4870-b1ff-ff37572f43a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:32.996857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fb4b-1138-4870-b1ff-ff37572f43a1 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:32.997200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fb4b-1138-4870-b1ff-ff37572f43a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:32.997532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fb4b-1138-4870-b1ff-ff37572f43a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:33.007083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648fb4b-1138-4870-b1ff-ff37572f43a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:33.009400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648fb4b-1138-4870-b1ff-ff37572f43a1 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:33.017937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1212/2425] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:21:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:21:33.787396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4facc-a367-46ca-b029-e8f806eec56d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:33.788217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4facc-a367-46ca-b029-e8f806eec56d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:33.788217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4facc-a367-46ca-b029-e8f806eec56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:33.864961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-edd4facc-a367-46ca-b029-e8f806eec56d None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:33.869405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-edd4facc-a367-46ca-b029-e8f806eec56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:33.885197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1214/2426] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 31 12:21:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:21:33.904558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fb4b-1138-4870-b1ff-ff37572f43a1 service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:33.920967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fb4b-1138-4870-b1ff-ff37572f43a1 service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:33.929131 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7391a7c-cd65-4de8-aad4-4e5d175f4351 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:33.929754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7391a7c-cd65-4de8-aad4-4e5d175f4351 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:33.930093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7391a7c-cd65-4de8-aad4-4e5d175f4351 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:33.930426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7391a7c-cd65-4de8-aad4-4e5d175f4351 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:33.939952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7391a7c-cd65-4de8-aad4-4e5d175f4351 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:33.941667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7391a7c-cd65-4de8-aad4-4e5d175f4351 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:33.952264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1213/2427] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:21:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:21:34.250698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864840b3-f278-427a-b3f3-9ac23b847048 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:34.251049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864840b3-f278-427a-b3f3-9ac23b847048 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:34.251504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864840b3-f278-427a-b3f3-9ac23b847048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:34.276729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-864840b3-f278-427a-b3f3-9ac23b847048 None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:34.282480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-864840b3-f278-427a-b3f3-9ac23b847048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:34.297271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1215/2428] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:21:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:34.321776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7391a7c-cd65-4de8-aad4-4e5d175f4351 service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:34.339168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7391a7c-cd65-4de8-aad4-4e5d175f4351 service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:34.357441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900e344-4342-4846-8cb1-0c4d04fdd297 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:34.358504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900e344-4342-4846-8cb1-0c4d04fdd297 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:34.358926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900e344-4342-4846-8cb1-0c4d04fdd297 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:34.359367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900e344-4342-4846-8cb1-0c4d04fdd297 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:34.373851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900e344-4342-4846-8cb1-0c4d04fdd297 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:34.376151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900e344-4342-4846-8cb1-0c4d04fdd297 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:34.391487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1214/2429] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:21:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:21:34.961381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75e3d-8db4-403d-9ad5-8a013bf7154e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:34.962807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75e3d-8db4-403d-9ad5-8a013bf7154e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:34.963220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75e3d-8db4-403d-9ad5-8a013bf7154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:34.997549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bde75e3d-8db4-403d-9ad5-8a013bf7154e None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:34.999638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bde75e3d-8db4-403d-9ad5-8a013bf7154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:35.015216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1216/2430] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 31 12:21:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:35.029168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900e344-4342-4846-8cb1-0c4d04fdd297 service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:35.042552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900e344-4342-4846-8cb1-0c4d04fdd297 service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:35.048750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a08b-92da-4621-970e-4071467a5dfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:35.049443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a08b-92da-4621-970e-4071467a5dfb service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:35.049805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a08b-92da-4621-970e-4071467a5dfb service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:35.050085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a08b-92da-4621-970e-4071467a5dfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:35.061591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97a08b-92da-4621-970e-4071467a5dfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:35.063909 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97a08b-92da-4621-970e-4071467a5dfb service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:35.077930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1215/2431] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:21:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:21:36.588079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55330abf-746d-4506-a7b2-dd457426dbb7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:36.588622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55330abf-746d-4506-a7b2-dd457426dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:36.590546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55330abf-746d-4506-a7b2-dd457426dbb7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:36.591397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1217/2432] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:21:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:21:36.606900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4d705-6c18-454b-b574-c5492ac89593 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:36.607973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4d705-6c18-454b-b574-c5492ac89593 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:36.608269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4d705-6c18-454b-b574-c5492ac89593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:36.719990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4da4d705-6c18-454b-b574-c5492ac89593 None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:36.723146 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4da4d705-6c18-454b-b574-c5492ac89593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:36.878722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1216/2433] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:21:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:21:36.917316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55330abf-746d-4506-a7b2-dd457426dbb7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:36.922630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-55330abf-746d-4506-a7b2-dd457426dbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:37.039074 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55330abf-746d-4506-a7b2-dd457426dbb7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:37.043385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e0e13-647f-4ef9-af9b-12d0b1b0f4fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:37.047653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e0e13-647f-4ef9-af9b-12d0b1b0f4fb service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:37.047653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e0e13-647f-4ef9-af9b-12d0b1b0f4fb service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:37.047653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e0e13-647f-4ef9-af9b-12d0b1b0f4fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:37.066223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e0e13-647f-4ef9-af9b-12d0b1b0f4fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:37.068437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e0e13-647f-4ef9-af9b-12d0b1b0f4fb service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:37.083367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1218/2434] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:37.199456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e274a-57e1-41bf-b78c-5f59a6d5809f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:37.199735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e274a-57e1-41bf-b78c-5f59a6d5809f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:37.200002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e274a-57e1-41bf-b78c-5f59a6d5809f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:37.201606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1217/2435] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:21:37.208221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1d05-db1d-4b2f-aa0f-76905cb33774 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:37.208813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1d05-db1d-4b2f-aa0f-76905cb33774 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:37.209311 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1d05-db1d-4b2f-aa0f-76905cb33774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:37.232532 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-363c1d05-db1d-4b2f-aa0f-76905cb33774 None None] MFA Rules not processed for user `ba8c7348ad2e457e80e1fdd2d93aacc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:37.233796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-363c1d05-db1d-4b2f-aa0f-76905cb33774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:37.318288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1219/2436] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:21:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:37.332333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e274a-57e1-41bf-b78c-5f59a6d5809f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:37.334941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-885e274a-57e1-41bf-b78c-5f59a6d5809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:37.350002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e274a-57e1-41bf-b78c-5f59a6d5809f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:37.354420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3c376-3016-4b54-9d9a-4382436502df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:37.355088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3c376-3016-4b54-9d9a-4382436502df service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:37.355417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3c376-3016-4b54-9d9a-4382436502df service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:37.355690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3c376-3016-4b54-9d9a-4382436502df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:37.363713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3c376-3016-4b54-9d9a-4382436502df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:37.365441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3c376-3016-4b54-9d9a-4382436502df service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:37.384258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1218/2437] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:37.486233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bea30-887a-4293-a398-0308ee1d669b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:37.486549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bea30-887a-4293-a398-0308ee1d669b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:37.486890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bea30-887a-4293-a398-0308ee1d669b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:37.488697 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1220/2438] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 31 12:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 12:21:37.496068 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64324-2ca2-46b6-a3fe-109d92350c8f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:37.496516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64324-2ca2-46b6-a3fe-109d92350c8f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:37.496890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64324-2ca2-46b6-a3fe-109d92350c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:37.520411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cdf64324-2ca2-46b6-a3fe-109d92350c8f None None] MFA Rules not processed for user `0b113316bc3348b5a12c1d7b616bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:37.521740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cdf64324-2ca2-46b6-a3fe-109d92350c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:37.531810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1219/2439] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 31 12:21:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:37.542662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bea30-887a-4293-a398-0308ee1d669b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:37.556122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bea30-887a-4293-a398-0308ee1d669b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:37.562870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:37.564888 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:37.565378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:37.566005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:37.603505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:37.614113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:37.631558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1221/2440] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:37.706605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf64324-2ca2-46b6-a3fe-109d92350c8f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:37.708100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cdf64324-2ca2-46b6-a3fe-109d92350c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:37.759667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:37.861187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf64324-2ca2-46b6-a3fe-109d92350c8f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:37.869055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4967c-ea81-42b2-98d7-0c5e44687da7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:37.869762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4967c-ea81-42b2-98d7-0c5e44687da7 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:37.870145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4967c-ea81-42b2-98d7-0c5e44687da7 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:37.870672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4967c-ea81-42b2-98d7-0c5e44687da7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:37.880556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ef4967c-ea81-42b2-98d7-0c5e44687da7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:37.913268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4967c-ea81-42b2-98d7-0c5e44687da7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:37.914901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4967c-ea81-42b2-98d7-0c5e44687da7 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:37.930893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1220/2441] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:38.209098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:38.211788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:38.271459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:38.378253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040c877-d846-4ec6-92fa-cd01b5e01346 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:38.390528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:38.392346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:38.392824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:38.393478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] PATH_INFO: `/v3/users/7d60bea32ca4476d84d00d9ff95f3516` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:38.395465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d60bea32ca4476d84d00d9ff95f3516)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:38.398393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:38.520472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d60bea32ca4476d84d00d9ff95f3516'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:38.521173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d60bea32ca4476d84d00d9ff95f3516'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:38.539613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d60bea32ca4476d84d00d9ff95f3516'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:38.554211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d60bea32ca4476d84d00d9ff95f3516'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:38.568449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d60bea32ca4476d84d00d9ff95f3516'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:38.584728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1222/2442] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:21:38 2022] DELETE /identity/v3/users/7d60bea32ca4476d84d00d9ff95f3516 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:38.590879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de59237-29f3-47ad-a903-358bc487890d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:38.591222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de59237-29f3-47ad-a903-358bc487890d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:38.592193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de59237-29f3-47ad-a903-358bc487890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:38.622399 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0de59237-29f3-47ad-a903-358bc487890d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:38.624625 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0de59237-29f3-47ad-a903-358bc487890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:38.762807 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1221/2443] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:38.774750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:38.776252 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:38.868991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4601607-28fe-4225-926f-71d42cade9a5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:38.873860 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef926ebe-dd44-4a8f-9542-7ee7caf03f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:38.874501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926ebe-dd44-4a8f-9542-7ee7caf03f33 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:38.874783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926ebe-dd44-4a8f-9542-7ee7caf03f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:38.875057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926ebe-dd44-4a8f-9542-7ee7caf03f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:38.894130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef926ebe-dd44-4a8f-9542-7ee7caf03f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:38.894130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef926ebe-dd44-4a8f-9542-7ee7caf03f33 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:38.908424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1223/2444] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:21:39.159600 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de59237-29f3-47ad-a903-358bc487890d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:39.165132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0de59237-29f3-47ad-a903-358bc487890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:39.183719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de59237-29f3-47ad-a903-358bc487890d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:39.192863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:39.202369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:39.202951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:39.203265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] PATH_INFO: `/v3/projects/90897266779e40028dc2be9301efff53` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:39.204674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90897266779e40028dc2be9301efff53)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:39.207076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:39.293010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90897266779e40028dc2be9301efff53', 'request_id': 'req-a8172377-c471-456f-b6ae-fe06f94f6ee3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:39.305183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90897266779e40028dc2be9301efff53', 'request_id': 'req-a8172377-c471-456f-b6ae-fe06f94f6ee3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:39.317382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90897266779e40028dc2be9301efff53', 'request_id': 'req-a8172377-c471-456f-b6ae-fe06f94f6ee3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:39.331785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] The token cache is being invalidate because project 90897266779e40028dc2be9301efff53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:21:39.332081 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:39.332748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:21:39.334396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1222/2445] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:21:39 2022] DELETE /identity/v3/projects/90897266779e40028dc2be9301efff53 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:21:41.613080 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fdc9c-1be4-4428-b624-15896f068bdd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:41.613981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fdc9c-1be4-4428-b624-15896f068bdd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:41.614129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fdc9c-1be4-4428-b624-15896f068bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:41.636294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-577fdc9c-1be4-4428-b624-15896f068bdd None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:21:41.637615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-577fdc9c-1be4-4428-b624-15896f068bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:41.682436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:41.698764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1224/2446] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:21:41.711538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:21:41.713192 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:21:41.758953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:41.800182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8172377-c471-456f-b6ae-fe06f94f6ee3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:21:41.809119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab901-2952-4407-b355-2c04e4a995d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:21:41.809753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab901-2952-4407-b355-2c04e4a995d4 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:21:41.810046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab901-2952-4407-b355-2c04e4a995d4 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:21:41.810301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab901-2952-4407-b355-2c04e4a995d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:21:41.821815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ab901-2952-4407-b355-2c04e4a995d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:21:41.824510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ab901-2952-4407-b355-2c04e4a995d4 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:21:41.835173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1223/2447] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:21:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:21:51.695886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:21:51.830394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=16 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:22:16.356748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021205-98ec-4689-b65a-8162d557103e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:22:16.357287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021205-98ec-4689-b65a-8162d557103e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:22:16.357287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021205-98ec-4689-b65a-8162d557103e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:22:16.378686 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ca021205-98ec-4689-b65a-8162d557103e None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:22:16.379983 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ca021205-98ec-4689-b65a-8162d557103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:22:16.389055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1225/2448] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 31 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:22:16.415620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab901-2952-4407-b355-2c04e4a995d4 service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:22:16.428733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab901-2952-4407-b355-2c04e4a995d4 service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:22:16.434623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d80c4-f78d-4885-85ce-0ef806ec9e9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:22:16.435342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d80c4-f78d-4885-85ce-0ef806ec9e9b service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:22:16.435738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d80c4-f78d-4885-85ce-0ef806ec9e9b service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:22:16.436027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d80c4-f78d-4885-85ce-0ef806ec9e9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:22:16.446208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d80c4-f78d-4885-85ce-0ef806ec9e9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:22:16.447860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d80c4-f78d-4885-85ce-0ef806ec9e9b service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:22:16.457028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1224/2449] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:22:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:22:26.371928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:22:26.453045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:22:34.092579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044020a-5d1b-4603-83d6-307c660e7fc2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:22:34.093334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044020a-5d1b-4603-83d6-307c660e7fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:22:34.093334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044020a-5d1b-4603-83d6-307c660e7fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:22:34.122483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4044020a-5d1b-4603-83d6-307c660e7fc2 None None] MFA Rules not processed for user `37a3870e9df04d7795a649a21b722801`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:22:34.123880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4044020a-5d1b-4603-83d6-307c660e7fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:22:34.213755 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1226/2450] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:22:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:22:44.199933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=25 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:22:54.239162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c2dc-c80e-4ab0-af4f-e7e2a5c65ba2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:22:54.242820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c2dc-c80e-4ab0-af4f-e7e2a5c65ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:22:54.242820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c2dc-c80e-4ab0-af4f-e7e2a5c65ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:22:54.263020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5906c2dc-c80e-4ab0-af4f-e7e2a5c65ba2 None None] MFA Rules not processed for user `4c3e5c1776b8470a9b677bac8dfbc9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:22:54.264289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5906c2dc-c80e-4ab0-af4f-e7e2a5c65ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:22:54.329291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1225/2451] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:23:04.319824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=25 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:23:29.465877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044020a-5d1b-4603-83d6-307c660e7fc2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:23:29.468308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4044020a-5d1b-4603-83d6-307c660e7fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:29.590575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044020a-5d1b-4603-83d6-307c660e7fc2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:23:29.598663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:23:29.599784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:23:29.600385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:23:29.600758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:23:29.604296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:29.752890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:23:29.756331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:23:29.778695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1227/2452] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:23:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:23:33.320799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906c2dc-c80e-4ab0-af4f-e7e2a5c65ba2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:23:33.343404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906c2dc-c80e-4ab0-af4f-e7e2a5c65ba2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:23:33.352062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:23:33.353272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:23:33.353780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:23:33.354261 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:23:33.357608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:33.384828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:23:33.387190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:23:33.403055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1226/2453] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:23:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:23:36.339103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:23:36.340920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:36.358827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcee6f0-5364-49c8-bdff-b74770df2823 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:23:36.368470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:23:36.369474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:23:36.369910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:23:36.370354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:23:36.373622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:36.450007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:23:36.453436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:23:36.477975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1228/2454] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:23:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:23:43.396454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:23:46.466858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=82 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:23:51.230894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:23:51.249940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2919-a2e9-4faa-9810-65d42e563669 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:23:51.258835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:23:51.259847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:23:51.260273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:23:51.260665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:23:51.263676 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:51.284350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:23:51.287182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:23:51.304504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1227/2455] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:23:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:23:51.332673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:23:51.334608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:51.441430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:23:51.446336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:51.450322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab29180-1c79-476f-873f-bcdac2093848 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:23:51.460708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:23:51.461930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:23:51.462501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:23:51.463062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:23:51.467711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:51.484455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d224-9ae4-4756-a065-383d872f879b service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:23:51.500809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f270a4f-e87b-48a8-89b6-4de001e7c542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:23:51.502827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f270a4f-e87b-48a8-89b6-4de001e7c542 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:23:51.503569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f270a4f-e87b-48a8-89b6-4de001e7c542 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:23:51.504028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f270a4f-e87b-48a8-89b6-4de001e7c542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:23:51.506471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5f270a4f-e87b-48a8-89b6-4de001e7c542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:51.614388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:23:51.620337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:23:51.636698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f270a4f-e87b-48a8-89b6-4de001e7c542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:23:51.638632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f270a4f-e87b-48a8-89b6-4de001e7c542 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:23:51.658357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1228/2456] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:23:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:23:51.681220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1229/2457] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:23:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:23:56.987686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15972189-43cb-4864-9ee4-b3fe19d5c031 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:23:56.988890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15972189-43cb-4864-9ee4-b3fe19d5c031 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:23:56.988890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15972189-43cb-4864-9ee4-b3fe19d5c031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:23:57.022440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-15972189-43cb-4864-9ee4-b3fe19d5c031 None None] MFA Rules not processed for user `a8320be5e9644b6699ac526913e109eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:23:57.024894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-15972189-43cb-4864-9ee4-b3fe19d5c031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:23:57.038593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1229/2458] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:23:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:24:01.668741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=48 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:24:07.012503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=37 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:24:43.395300 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:24:43.409949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22de47a-5694-4001-977b-15335ae0e3db service glance] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:24:43.415296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:24:43.416216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:24:43.416565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:24:43.416892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:24:43.421101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:24:43.434999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:24:43.436643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:24:43.447392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1230/2459] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:24:53.442361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:25:00.478841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15972189-43cb-4864-9ee4-b3fe19d5c031 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:25:00.497366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15972189-43cb-4864-9ee4-b3fe19d5c031 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:25:00.504692 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:25:00.505553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:25:00.505977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:25:00.506422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:25:00.512911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:25:00.607888 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:25:00.610346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:25:00.623471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1230/2460] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:25:10.616865 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=28 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:25:51.596150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:25:51.598222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:25:51.702124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8ccac-ef78-4721-af25-a6d41b926a2c service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:25:51.708909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:25:51.709939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:25:51.710405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:25:51.710867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:25:51.714667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:25:51.735761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:25:51.738304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:25:51.755862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1231/2461] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:25:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:26:01.747705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=26 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:26:38.056623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:26:38.058621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:26:38.328085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf803989-f86f-4c52-8c8f-b5b69b5d5a1a service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:26:38.337197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:26:38.338189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:26:38.338667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:26:38.339109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:26:38.342199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:26:38.361450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:26:38.364466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:26:38.377944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1231/2462] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:26:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:26:48.372977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=63 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:28:36.740927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:28:36.816485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c8b18-0f7e-4866-a443-4603d9cd296a service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:28:36.825405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ca363-3f15-41d6-acc5-a2bec47dda9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:28:36.826269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ca363-3f15-41d6-acc5-a2bec47dda9a service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:28:36.826584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ca363-3f15-41d6-acc5-a2bec47dda9a service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:28:36.826895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ca363-3f15-41d6-acc5-a2bec47dda9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:28:36.838704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ca363-3f15-41d6-acc5-a2bec47dda9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:28:36.840648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ca363-3f15-41d6-acc5-a2bec47dda9a service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:28:36.877530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1232/2463] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:28:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:28:46.873536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=31 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:28:50.026892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:28:50.072278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914e1eb-af58-4304-97bf-a2e6be3a3f72 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:28:50.080338 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba706446-e08d-4a7b-aafd-b88a2cdbb440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:28:50.081379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba706446-e08d-4a7b-aafd-b88a2cdbb440 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:28:50.081869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba706446-e08d-4a7b-aafd-b88a2cdbb440 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:28:50.082482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba706446-e08d-4a7b-aafd-b88a2cdbb440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:28:50.096270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba706446-e08d-4a7b-aafd-b88a2cdbb440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:28:50.097848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba706446-e08d-4a7b-aafd-b88a2cdbb440 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:28:50.110963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1232/2464] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:28:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:28:51.040594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ca363-3f15-41d6-acc5-a2bec47dda9a service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:28:51.056362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ca363-3f15-41d6-acc5-a2bec47dda9a service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:28:51.064940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:28:51.066213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:28:51.066544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:28:51.067016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:28:51.081667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:28:51.083940 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:28:51.131528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1233/2465] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:28:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:29:00.104294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=10 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:29:01.126764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=13 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:29:01.973915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba706446-e08d-4a7b-aafd-b88a2cdbb440 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:29:01.992194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba706446-e08d-4a7b-aafd-b88a2cdbb440 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:29:01.999795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a30e-6c05-4e08-97e6-0ed4402e2069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:29:02.000818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a30e-6c05-4e08-97e6-0ed4402e2069 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:29:02.001731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a30e-6c05-4e08-97e6-0ed4402e2069 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:29:02.002026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a30e-6c05-4e08-97e6-0ed4402e2069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:29:02.018220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9a30e-6c05-4e08-97e6-0ed4402e2069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:29:02.019729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9a30e-6c05-4e08-97e6-0ed4402e2069 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:29:02.033495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1233/2466] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:29:12.026620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:29:59.928621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:29:59.931114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:29:59.972872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5a267-0cc9-4ff6-85e4-c7a778d6a5b4 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:29:59.979600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab18b05-b186-48ee-9a03-d3dae3c78b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:29:59.981189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab18b05-b186-48ee-9a03-d3dae3c78b9f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:29:59.981692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab18b05-b186-48ee-9a03-d3dae3c78b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:29:59.982184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab18b05-b186-48ee-9a03-d3dae3c78b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:29:59.998890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab18b05-b186-48ee-9a03-d3dae3c78b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:00.001295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab18b05-b186-48ee-9a03-d3dae3c78b9f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:00.048190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1234/2467] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:29:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:30:10.045714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=18 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:30:38.981350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a30e-6c05-4e08-97e6-0ed4402e2069 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:30:39.002719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a30e-6c05-4e08-97e6-0ed4402e2069 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:30:39.016605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:30:39.017401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:39.018239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:39.018845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:39.021577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:30:39.058698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:39.061059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:39.080595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1234/2468] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:30:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:30:39.537338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab18b05-b186-48ee-9a03-d3dae3c78b9f service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:30:39.632226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab18b05-b186-48ee-9a03-d3dae3c78b9f service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:30:39.647206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c77acb-f9b3-42db-acc7-1d37546d6f37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:30:39.652541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77acb-f9b3-42db-acc7-1d37546d6f37 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:39.653579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77acb-f9b3-42db-acc7-1d37546d6f37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:39.654681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77acb-f9b3-42db-acc7-1d37546d6f37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:39.699292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c77acb-f9b3-42db-acc7-1d37546d6f37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:39.699292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c77acb-f9b3-42db-acc7-1d37546d6f37 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:39.712084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:30:39.718244 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1235/2469] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:30:39.728580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286081b-4b5d-43e2-b6ac-8d7c310fca5d service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:30:39.735519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc78e3a-73a9-4cf8-8110-eeab66bb07a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:30:39.736757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc78e3a-73a9-4cf8-8110-eeab66bb07a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:39.737054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc78e3a-73a9-4cf8-8110-eeab66bb07a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:39.737432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc78e3a-73a9-4cf8-8110-eeab66bb07a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:39.752956 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc78e3a-73a9-4cf8-8110-eeab66bb07a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:39.756542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc78e3a-73a9-4cf8-8110-eeab66bb07a3 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:39.786491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1235/2470] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:30:44.068422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af276-d3fb-41f8-9adc-31129462508e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:44.069306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af276-d3fb-41f8-9adc-31129462508e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:44.069306 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af276-d3fb-41f8-9adc-31129462508e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:44.164829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6f6af276-d3fb-41f8-9adc-31129462508e None None] MFA Rules not processed for user `4c3e5c1776b8470a9b677bac8dfbc9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:30:44.167892 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6f6af276-d3fb-41f8-9adc-31129462508e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:30:44.215746 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1236/2471] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:30:44.674365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc78e3a-73a9-4cf8-8110-eeab66bb07a3 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:30:44.687168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc78e3a-73a9-4cf8-8110-eeab66bb07a3 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:30:44.691593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:30:44.692342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:44.692503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:44.692775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:44.694727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:30:44.738747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:44.741640 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:44.758578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1236/2472] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:30:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:30:45.222655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6af276-d3fb-41f8-9adc-31129462508e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:30:45.250237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6af276-d3fb-41f8-9adc-31129462508e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:30:45.261475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:30:45.262540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:45.263084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:45.263611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:45.267307 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:30:45.302566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:45.305580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:45.325523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1237/2473] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:30:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:30:51.846701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:30:51.868652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df19582-0afd-4272-a499-5eefeee4ca53 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:30:51.890738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d99c0-f6aa-40b7-a667-4de4d946db86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:30:51.890738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d99c0-f6aa-40b7-a667-4de4d946db86 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:51.890738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d99c0-f6aa-40b7-a667-4de4d946db86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:51.890738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d99c0-f6aa-40b7-a667-4de4d946db86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:51.918580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d99c0-f6aa-40b7-a667-4de4d946db86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:51.921483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d99c0-f6aa-40b7-a667-4de4d946db86 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:51.984862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1237/2474] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:30:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:30:54.524475 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:30:54.552412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8f2d5-bced-4361-8523-520387cf26fa service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:30:54.566960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:30:54.570311 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:54.570874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:54.571472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:54.587594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:30:54.628836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:54.633663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:54.667441 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1238/2475] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:30:59.329022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d99c0-f6aa-40b7-a667-4de4d946db86 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:30:59.354060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d99c0-f6aa-40b7-a667-4de4d946db86 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:30:59.367282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:30:59.367627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:30:59.368092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:30:59.368592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:30:59.371607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:30:59.396670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:30:59.400057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:30:59.422690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1238/2476] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:31:04.652012 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=41 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:31:09.412576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=47 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:31:44.916899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:31:44.935033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ef86f-dea9-4c62-a7b6-dddc85d56c6a service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:31:44.941894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f80468-ccfd-4de2-bafa-5a007f3ada92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:31:44.942577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f80468-ccfd-4de2-bafa-5a007f3ada92 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:31:44.942900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f80468-ccfd-4de2-bafa-5a007f3ada92 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:31:44.943204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f80468-ccfd-4de2-bafa-5a007f3ada92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:31:44.955633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f80468-ccfd-4de2-bafa-5a007f3ada92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:31:44.957411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f80468-ccfd-4de2-bafa-5a007f3ada92 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:31:44.972744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1239/2477] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:31:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:31:54.966277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:33:36.255413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:33:36.272775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d8b1e-077f-4761-bbcd-d7069e5f005d service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:33:36.279220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b35649-f369-4011-b68f-312ef7adf1ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:33:36.280201 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b35649-f369-4011-b68f-312ef7adf1ef service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:33:36.280626 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b35649-f369-4011-b68f-312ef7adf1ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:33:36.281057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b35649-f369-4011-b68f-312ef7adf1ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:33:36.295174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b35649-f369-4011-b68f-312ef7adf1ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:33:36.296911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b35649-f369-4011-b68f-312ef7adf1ef service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:33:36.311721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1239/2478] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:33:46.303811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:33:50.407701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f80468-ccfd-4de2-bafa-5a007f3ada92 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:33:50.422595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f80468-ccfd-4de2-bafa-5a007f3ada92 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:33:50.428822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edbca0-6424-43b7-9072-ffd912ecb862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:33:50.429910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edbca0-6424-43b7-9072-ffd912ecb862 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:33:50.430392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edbca0-6424-43b7-9072-ffd912ecb862 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:33:50.430868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edbca0-6424-43b7-9072-ffd912ecb862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:33:50.534681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70edbca0-6424-43b7-9072-ffd912ecb862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:33:50.536978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70edbca0-6424-43b7-9072-ffd912ecb862 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:33:50.553724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1240/2479] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:33:51.811257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b35649-f369-4011-b68f-312ef7adf1ef service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:33:51.824248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b35649-f369-4011-b68f-312ef7adf1ef service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:33:51.828662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:33:51.829728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:33:51.830118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:33:51.830502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:33:51.833011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:33:51.931655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:33:51.933179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:33:51.947763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1240/2480] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:33:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:34:00.546077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=28 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:34:01.942019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=33 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:35:00.562420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edbca0-6424-43b7-9072-ffd912ecb862 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:35:00.572747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edbca0-6424-43b7-9072-ffd912ecb862 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:35:00.577921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824de4d7-b8d3-4067-b77a-7fb40fc11c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:35:00.578759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824de4d7-b8d3-4067-b77a-7fb40fc11c80 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:35:00.579151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824de4d7-b8d3-4067-b77a-7fb40fc11c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:35:00.579599 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824de4d7-b8d3-4067-b77a-7fb40fc11c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:35:00.591277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824de4d7-b8d3-4067-b77a-7fb40fc11c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:35:00.593353 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824de4d7-b8d3-4067-b77a-7fb40fc11c80 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:35:00.668618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1241/2481] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:35:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:35:01.316691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:35:01.318608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:35:01.405761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dea81c-f6fb-4b5c-be61-669e932c63d6 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:35:01.410288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:35:01.411046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:35:01.411408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:35:01.411799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:35:01.414245 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:35:01.505285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:35:01.507091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:35:01.525428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1241/2482] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:35:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:35:10.662776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=28 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:35:11.516525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=54 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:35:39.425294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824de4d7-b8d3-4067-b77a-7fb40fc11c80 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:35:39.446585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824de4d7-b8d3-4067-b77a-7fb40fc11c80 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:35:39.454471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:35:39.455488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:35:39.455916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:35:39.456344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:35:39.460160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:35:39.487710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:35:39.489360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:35:39.509031 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1242/2483] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:35:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:35:43.569125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:35:43.580985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864448b3-6eb2-44b8-a2aa-ad41b2a4b442 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:35:43.585355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:35:43.585984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:35:43.586260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:35:43.586528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:35:43.588520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:35:43.599123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:35:43.600801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:35:43.610683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1242/2484] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:35:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:35:49.498751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:35:51.452749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:35:51.454505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:35:51.470738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5938786-ea9d-450c-9488-5dda1e51c7a4 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:35:51.477550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58027bae-f4f4-4051-818d-d63178b28884 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:35:51.479453 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027bae-f4f4-4051-818d-d63178b28884 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:35:51.479765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027bae-f4f4-4051-818d-d63178b28884 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:35:51.480031 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027bae-f4f4-4051-818d-d63178b28884 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:35:51.593351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58027bae-f4f4-4051-818d-d63178b28884 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:35:51.597411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58027bae-f4f4-4051-818d-d63178b28884 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:35:51.617092 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1243/2485] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:35:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:35:53.606160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:36:01.608220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=28 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:36:50.839212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:36:50.841014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:36:51.000102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f23242-f6a5-467c-8694-f5dbab62609b service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:36:51.006383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:36:51.007017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:36:51.007325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:36:51.007597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:36:51.009618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:36:51.023121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:36:51.024863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:36:51.036570 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1243/2486] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:36:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:37:01.031559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=52 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:37:55.911290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58027bae-f4f4-4051-818d-d63178b28884 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:37:55.939808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58027bae-f4f4-4051-818d-d63178b28884 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:37:55.949147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:37:55.949955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:37:55.950323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:37:55.950642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:37:55.966988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:37:55.969424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:37:55.998824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1244/2487] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:37:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:38:05.979563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=9 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:38:35.687634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:38:35.706304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da96420-ae4a-4a69-94e6-ff5fd54110ab service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:38:35.713920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:38:35.715183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:38:35.715723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:38:35.716204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:38:35.720079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:38:35.867016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:38:35.869499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:38:35.887766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1244/2488] 10.222.0.27 () {60 vars in 1396 bytes} [Wed Aug 31 12:38:35 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:38:45.882482 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=33 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:38:52.129963 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:38:52.132011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:38:52.236358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713e41c-1a2b-4b93-9847-feb8cfe4d5f4 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:38:52.248468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1f231-105a-4438-93f7-116c67c9570d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:38:52.250006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1f231-105a-4438-93f7-116c67c9570d service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:38:52.250819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1f231-105a-4438-93f7-116c67c9570d service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:38:52.251435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1f231-105a-4438-93f7-116c67c9570d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:38:52.269568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1f231-105a-4438-93f7-116c67c9570d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:38:52.272834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1f231-105a-4438-93f7-116c67c9570d service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:38:52.323153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1245/2489] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:38:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:39:02.317397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=31 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:39:03.853100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:39:03.855824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:39:04.016833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab1721-8720-42d6-95c8-d29ec5e96926 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:39:04.021897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:39:04.022720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:39:04.023023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:39:04.023478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:39:04.025784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:39:04.044458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:39:04.046524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:39:04.065323 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1245/2490] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:39:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:39:14.054336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=35 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:40:00.387946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1f231-105a-4438-93f7-116c67c9570d service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:40:00.406691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1f231-105a-4438-93f7-116c67c9570d service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:40:00.413778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:40:00.414418 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:40:00.414698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:40:00.414975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:40:00.417132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:40:00.467352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:40:00.470835 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:40:00.507970 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1246/2491] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:40:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:40:00.796734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:40:00.810783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5a829-f5d6-48b1-a233-b8e556488f23 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:40:00.815486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca7bfd-415a-4665-a3a8-b561e056e5a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:40:00.816225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca7bfd-415a-4665-a3a8-b561e056e5a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:40:00.816642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca7bfd-415a-4665-a3a8-b561e056e5a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:40:00.816960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca7bfd-415a-4665-a3a8-b561e056e5a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:40:00.829673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca7bfd-415a-4665-a3a8-b561e056e5a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:40:00.831763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca7bfd-415a-4665-a3a8-b561e056e5a2 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:40:00.845905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1246/2492] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:40:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:40:10.505667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=19 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:40:10.839574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:40:38.815415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:40:38.832488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c871d-e7b8-4320-99da-41c2503555be service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:40:38.838821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aef4b2-7340-400d-850d-d3896ef52198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:40:38.839950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aef4b2-7340-400d-850d-d3896ef52198 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:40:38.840421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aef4b2-7340-400d-850d-d3896ef52198 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:40:38.840902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aef4b2-7340-400d-850d-d3896ef52198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:40:38.872834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aef4b2-7340-400d-850d-d3896ef52198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:40:38.876672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aef4b2-7340-400d-850d-d3896ef52198 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:40:38.898910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1247/2493] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:40:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:40:48.564488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca7bfd-415a-4665-a3a8-b561e056e5a2 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:40:48.628231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca7bfd-415a-4665-a3a8-b561e056e5a2 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:40:48.634575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396237b8-6c45-4858-ac2a-e352aa0473f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:40:48.635627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396237b8-6c45-4858-ac2a-e352aa0473f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:40:48.636061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396237b8-6c45-4858-ac2a-e352aa0473f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:40:48.636475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396237b8-6c45-4858-ac2a-e352aa0473f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:40:48.651173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396237b8-6c45-4858-ac2a-e352aa0473f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:40:48.654120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396237b8-6c45-4858-ac2a-e352aa0473f2 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:40:48.671034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1247/2494] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:40:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:40:48.889048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:40:51.184795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aef4b2-7340-400d-850d-d3896ef52198 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:40:51.203769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aef4b2-7340-400d-850d-d3896ef52198 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:40:51.213568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:40:51.214465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:40:51.214883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:40:51.215384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:40:51.218771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:40:51.262278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:40:51.265133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:40:51.283534 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1248/2495] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:40:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:40:58.664230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=10 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:01.273524 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=11 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:03.610798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91cd7f-4df7-4e00-94f0-b698cad0797e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:03.611631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91cd7f-4df7-4e00-94f0-b698cad0797e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:03.611801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91cd7f-4df7-4e00-94f0-b698cad0797e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:03.753257 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ce91cd7f-4df7-4e00-94f0-b698cad0797e None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:41:03.754571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ce91cd7f-4df7-4e00-94f0-b698cad0797e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:03.876920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1248/2496] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 31 12:41:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:41:03.952895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:03.956974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:04.127398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c2081-0b4f-4c3e-bbb1-895841bddde1 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:04.137452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df5b2a-24d9-4f25-8021-2903cdf1f157 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:04.138165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df5b2a-24d9-4f25-8021-2903cdf1f157 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:04.138451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df5b2a-24d9-4f25-8021-2903cdf1f157 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:04.138733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df5b2a-24d9-4f25-8021-2903cdf1f157 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:04.154055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df5b2a-24d9-4f25-8021-2903cdf1f157 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:04.157239 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df5b2a-24d9-4f25-8021-2903cdf1f157 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:04.174876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1249/2497] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:41:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:41:05.130115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91cd7f-4df7-4e00-94f0-b698cad0797e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:05.154087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91cd7f-4df7-4e00-94f0-b698cad0797e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:05.159614 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:05.160361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:05.160621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:05.161124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:05.163381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:05.279100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:05.281209 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:05.316723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1249/2498] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:41:05.858824 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7478d8-067d-4594-89bf-52d651515e54 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:05.859744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7478d8-067d-4594-89bf-52d651515e54 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:05.859744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7478d8-067d-4594-89bf-52d651515e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:05.915494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6c7478d8-067d-4594-89bf-52d651515e54 None None] MFA Rules not processed for user `09b76f14426c4241aa49812f06f35cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:41:05.916924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6c7478d8-067d-4594-89bf-52d651515e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:05.939385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1250/2499] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:41:06.693285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:06.702897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a0695-079c-458a-a3c2-cef5ef93c59d service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:06.707537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a5328-b0b7-4140-911f-bd93a23c3d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:06.708346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5328-b0b7-4140-911f-bd93a23c3d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:06.708768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5328-b0b7-4140-911f-bd93a23c3d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:06.709082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5328-b0b7-4140-911f-bd93a23c3d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:06.711226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-690a5328-b0b7-4140-911f-bd93a23c3d25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:06.738757 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a5328-b0b7-4140-911f-bd93a23c3d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:06.741026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a5328-b0b7-4140-911f-bd93a23c3d25 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:06.752646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1250/2500] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:41:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:41:07.988138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7478d8-067d-4594-89bf-52d651515e54 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:07.993691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6c7478d8-067d-4594-89bf-52d651515e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:08.018179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7478d8-067d-4594-89bf-52d651515e54 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:08.026725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862476c-40c9-4e59-b44f-bb7d59323e5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:08.028651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862476c-40c9-4e59-b44f-bb7d59323e5d service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:08.029634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862476c-40c9-4e59-b44f-bb7d59323e5d service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:08.030426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862476c-40c9-4e59-b44f-bb7d59323e5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:08.034655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2862476c-40c9-4e59-b44f-bb7d59323e5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:08.061447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862476c-40c9-4e59-b44f-bb7d59323e5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:08.063763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862476c-40c9-4e59-b44f-bb7d59323e5d service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:08.082904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1251/2501] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:41:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:41:10.171553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad735e-4356-4c80-8144-21a807fdd877 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.172536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad735e-4356-4c80-8144-21a807fdd877 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.172536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad735e-4356-4c80-8144-21a807fdd877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:10.185237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:10.283104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-70ad735e-4356-4c80-8144-21a807fdd877 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:41:10.285316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-70ad735e-4356-4c80-8144-21a807fdd877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:10.300990 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1251/2502] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:41:10.315895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffef0d-a37e-49fc-a419-12a6745e31f3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.316550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffef0d-a37e-49fc-a419-12a6745e31f3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.316807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffef0d-a37e-49fc-a419-12a6745e31f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:10.351059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-27ffef0d-a37e-49fc-a419-12a6745e31f3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:41:10.353166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-27ffef0d-a37e-49fc-a419-12a6745e31f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:10.374060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1252/2503] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:41:10.381161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad735e-4356-4c80-8144-21a807fdd877 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:10.404951 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad735e-4356-4c80-8144-21a807fdd877 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:10.413552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:10.414605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.415107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.415631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:10.416826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:41:10.417582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:10.421178 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:10.459735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1252/2504] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:41:10.465826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffef0d-a37e-49fc-a419-12a6745e31f3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:10.490201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffef0d-a37e-49fc-a419-12a6745e31f3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:10.496773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd0cef-ff5d-4161-bdc0-e60ae791592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:10.498126 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd0cef-ff5d-4161-bdc0-e60ae791592b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.500190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd0cef-ff5d-4161-bdc0-e60ae791592b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.500855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd0cef-ff5d-4161-bdc0-e60ae791592b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:10.502131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd0cef-ff5d-4161-bdc0-e60ae791592b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:10.511138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd0cef-ff5d-4161-bdc0-e60ae791592b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:10.584844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:10.593436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1253/2505] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:41:10 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:41:10.615487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe5c0f-21aa-4a3a-81c9-5dbf15dc7b0c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:10.715125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9f20a-8955-4931-b92a-928d14d5ef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:10.716098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9f20a-8955-4931-b92a-928d14d5ef6b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.716349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9f20a-8955-4931-b92a-928d14d5ef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.716631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9f20a-8955-4931-b92a-928d14d5ef6b admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:10.717363 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9f20a-8955-4931-b92a-928d14d5ef6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:10.719936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9f20a-8955-4931-b92a-928d14d5ef6b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:10.763744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1253/2506] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:41:10 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:41:10.769679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd0cef-ff5d-4161-bdc0-e60ae791592b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:10.784503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd0cef-ff5d-4161-bdc0-e60ae791592b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:10.794171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b93c2b-3117-4b3d-9337-b6ea429b15cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:10.795015 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b93c2b-3117-4b3d-9337-b6ea429b15cf admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.795530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b93c2b-3117-4b3d-9337-b6ea429b15cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.796275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b93c2b-3117-4b3d-9337-b6ea429b15cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:10.797904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b93c2b-3117-4b3d-9337-b6ea429b15cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:10.800300 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b93c2b-3117-4b3d-9337-b6ea429b15cf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:10.823327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1254/2507] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:41:10.828228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9f20a-8955-4931-b92a-928d14d5ef6b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:10.847566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9f20a-8955-4931-b92a-928d14d5ef6b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:10.856186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c9bc0-c830-49ce-8ed2-7eabd0b74272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:10.857228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c9bc0-c830-49ce-8ed2-7eabd0b74272 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.857633 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c9bc0-c830-49ce-8ed2-7eabd0b74272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.858031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c9bc0-c830-49ce-8ed2-7eabd0b74272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:10.859046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c9bc0-c830-49ce-8ed2-7eabd0b74272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:10.862049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c9bc0-c830-49ce-8ed2-7eabd0b74272 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:10.891034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1254/2508] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:41:10.897095 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b93c2b-3117-4b3d-9337-b6ea429b15cf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:10.915195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b93c2b-3117-4b3d-9337-b6ea429b15cf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:10.923622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:10.924611 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.925014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.925423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] PATH_INFO: `/v3/projects/a86e12cf23804019ad942a0a9fc245f0/users/d852fb6149224e0bbf3020bd68afb610/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:10.952377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a86e12cf23804019ad942a0a9fc245f0, user_id=d852fb6149224e0bbf3020bd68afb610, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:10.957683 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:10.981679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1255/2509] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:41:10 2022] PUT /identity/v3/projects/a86e12cf23804019ad942a0a9fc245f0/users/d852fb6149224e0bbf3020bd68afb610/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:41:10.988900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706069d6-b3e0-448b-ba80-f37208fd80a1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:10.989259 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706069d6-b3e0-448b-ba80-f37208fd80a1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:10.989551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706069d6-b3e0-448b-ba80-f37208fd80a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:11.018842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-706069d6-b3e0-448b-ba80-f37208fd80a1 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:41:11.020148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-706069d6-b3e0-448b-ba80-f37208fd80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:11.105759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1255/2510] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:41:11.120497 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:11.122731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:11.154025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:11.246969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3f77c-0594-426d-88ea-a3211225f8c7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:11.255747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139176f9-7434-4470-843b-55ad810e14bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:11.257445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139176f9-7434-4470-843b-55ad810e14bc service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:11.257933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139176f9-7434-4470-843b-55ad810e14bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:11.259314 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139176f9-7434-4470-843b-55ad810e14bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:11.277237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139176f9-7434-4470-843b-55ad810e14bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:11.280200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139176f9-7434-4470-843b-55ad810e14bc service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:11.300556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1256/2511] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:41:17.357310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61669177-1a74-4854-b00c-75f7e28c9344 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:17.358099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61669177-1a74-4854-b00c-75f7e28c9344 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:17.358099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61669177-1a74-4854-b00c-75f7e28c9344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:17.399114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-61669177-1a74-4854-b00c-75f7e28c9344 None None] MFA Rules not processed for user `d852fb6149224e0bbf3020bd68afb610`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:41:17.400788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-61669177-1a74-4854-b00c-75f7e28c9344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:17.445378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:17.544919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1256/2512] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:41:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:41:17.552761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139176f9-7434-4470-843b-55ad810e14bc service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:17.576401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139176f9-7434-4470-843b-55ad810e14bc service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:17.591504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8945e81-256e-4fa6-8324-883b82844037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:17.592586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945e81-256e-4fa6-8324-883b82844037 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:17.594037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945e81-256e-4fa6-8324-883b82844037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:17.594828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945e81-256e-4fa6-8324-883b82844037 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:17.596329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8945e81-256e-4fa6-8324-883b82844037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:17.605138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8945e81-256e-4fa6-8324-883b82844037 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:17.714407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61669177-1a74-4854-b00c-75f7e28c9344 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:17.715977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1257/2513] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:41:17 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:41:17.741261 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61669177-1a74-4854-b00c-75f7e28c9344 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:17.868376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c65c63-e16f-41dd-b513-236229922f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:17.869326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c65c63-e16f-41dd-b513-236229922f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:17.869722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c65c63-e16f-41dd-b513-236229922f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:17.870180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c65c63-e16f-41dd-b513-236229922f50 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:17.871584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c65c63-e16f-41dd-b513-236229922f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:17.875535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c65c63-e16f-41dd-b513-236229922f50 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:17.917339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1257/2514] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:41:17 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:41:17.922910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8945e81-256e-4fa6-8324-883b82844037 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:17.936218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8945e81-256e-4fa6-8324-883b82844037 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:17.945709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38bf5d-c237-447f-adbe-87f8c3f4705f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:17.946723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38bf5d-c237-447f-adbe-87f8c3f4705f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:17.947215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38bf5d-c237-447f-adbe-87f8c3f4705f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:17.947883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38bf5d-c237-447f-adbe-87f8c3f4705f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:17.949013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38bf5d-c237-447f-adbe-87f8c3f4705f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:17.952157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38bf5d-c237-447f-adbe-87f8c3f4705f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:17.972233 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1258/2515] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:41:17.980091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c65c63-e16f-41dd-b513-236229922f50 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:17.998736 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c65c63-e16f-41dd-b513-236229922f50 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:18.007694 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91aa4c-c004-45f8-9132-e27ed301659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:18.008899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91aa4c-c004-45f8-9132-e27ed301659e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:18.009392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91aa4c-c004-45f8-9132-e27ed301659e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:18.009843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91aa4c-c004-45f8-9132-e27ed301659e admin admin] PATH_INFO: `/v3/projects/600e1c00222740fb95fc38800d98a9ac/users/01fe9242286a4bcda6465e24f471f8bc/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:18.039801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91aa4c-c004-45f8-9132-e27ed301659e admin admin] RBAC: Authorizing `identity:create_grant(project_id=600e1c00222740fb95fc38800d98a9ac, user_id=01fe9242286a4bcda6465e24f471f8bc, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:18.045713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91aa4c-c004-45f8-9132-e27ed301659e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:18.074576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1258/2516] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:41:17 2022] PUT /identity/v3/projects/600e1c00222740fb95fc38800d98a9ac/users/01fe9242286a4bcda6465e24f471f8bc/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:41:18.082278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38bf5d-c237-447f-adbe-87f8c3f4705f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:18.099156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38bf5d-c237-447f-adbe-87f8c3f4705f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:18.222448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df02f-4ec9-4f20-a721-c5d845784cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:18.223913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df02f-4ec9-4f20-a721-c5d845784cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:18.224433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df02f-4ec9-4f20-a721-c5d845784cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:18.224908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df02f-4ec9-4f20-a721-c5d845784cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:18.225600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df02f-4ec9-4f20-a721-c5d845784cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:18.227528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df02f-4ec9-4f20-a721-c5d845784cb0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:18.244331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1259/2517] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:41:18.252511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91aa4c-c004-45f8-9132-e27ed301659e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:18.272179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91aa4c-c004-45f8-9132-e27ed301659e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:18.278678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb354a4f-19e1-42d0-bbe0-13cf6d1d5da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:18.279885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354a4f-19e1-42d0-bbe0-13cf6d1d5da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:18.280367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354a4f-19e1-42d0-bbe0-13cf6d1d5da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:18.280898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354a4f-19e1-42d0-bbe0-13cf6d1d5da0 admin admin] PATH_INFO: `/v3/projects/600e1c00222740fb95fc38800d98a9ac/users/01fe9242286a4bcda6465e24f471f8bc/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:18.283756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb354a4f-19e1-42d0-bbe0-13cf6d1d5da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600e1c00222740fb95fc38800d98a9ac, user_id=01fe9242286a4bcda6465e24f471f8bc, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:18.286834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb354a4f-19e1-42d0-bbe0-13cf6d1d5da0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:18.300787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1259/2518] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:41:18 2022] PUT /identity/v3/projects/600e1c00222740fb95fc38800d98a9ac/users/01fe9242286a4bcda6465e24f471f8bc/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:41:20.023198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df02f-4ec9-4f20-a721-c5d845784cb0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:20.173722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df02f-4ec9-4f20-a721-c5d845784cb0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:20.199813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:20.304118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:20.305579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:20.305579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:20.305579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:20.308767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:20.354256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:20.355938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:20.388275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1260/2519] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:41:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:41:25.849503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590522-f2b8-46e0-83fa-4c13152eaedc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:25.851168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590522-f2b8-46e0-83fa-4c13152eaedc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:25.851758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590522-f2b8-46e0-83fa-4c13152eaedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:25.920832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b0590522-f2b8-46e0-83fa-4c13152eaedc None None] MFA Rules not processed for user `01fe9242286a4bcda6465e24f471f8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:41:25.922481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b0590522-f2b8-46e0-83fa-4c13152eaedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:26.094112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:26.100191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1260/2520] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:41:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:41:26.118331 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:26.140895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cabb3c-8ebb-42aa-9db4-ea29caf1a340 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:26.149171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da363a2a-1865-4006-8323-7d4e276583d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:26.150259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da363a2a-1865-4006-8323-7d4e276583d2 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:26.150713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da363a2a-1865-4006-8323-7d4e276583d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:26.150990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da363a2a-1865-4006-8323-7d4e276583d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:26.255498 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da363a2a-1865-4006-8323-7d4e276583d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:26.257187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da363a2a-1865-4006-8323-7d4e276583d2 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:26.269440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1261/2521] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:41:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:41:26.884370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0590522-f2b8-46e0-83fa-4c13152eaedc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:26.911368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0590522-f2b8-46e0-83fa-4c13152eaedc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:26.937874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:26.938654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:26.938938 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:26.940950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:26.946840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:26.974127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:26.977737 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:27.026960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1261/2522] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:41:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:41:28.133759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee55a15-975f-4be3-be56-3c784ffcc106 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:28.134936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee55a15-975f-4be3-be56-3c784ffcc106 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:28.135494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee55a15-975f-4be3-be56-3c784ffcc106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:28.160120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-eee55a15-975f-4be3-be56-3c784ffcc106 None None] MFA Rules not processed for user `d852fb6149224e0bbf3020bd68afb610`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:41:28.161602 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eee55a15-975f-4be3-be56-3c784ffcc106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:41:28.178786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1262/2523] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:41:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:41:36.992437 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:38.152988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=62 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:41:58.946049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:41:59.080931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1353dac-91ec-4b20-a04b-54e38a2b89e1 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:41:59.087562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:41:59.088579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:41:59.088826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:41:59.089238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:41:59.104138 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:41:59.106281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:41:59.120715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1262/2524] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:41:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:42:09.114107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=32 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:43:05.350581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee55a15-975f-4be3-be56-3c784ffcc106 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:43:05.394161 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee55a15-975f-4be3-be56-3c784ffcc106 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:43:05.411678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:43:05.415171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:43:05.416436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:43:05.417729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:43:05.428296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:43:05.472865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:43:05.484019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:43:05.531341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1263/2525] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:43:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:43:15.509367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:43:54.013902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:43:54.016786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:43:54.116634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55dd85-82ba-4a7a-abc8-3131a86b8e4f service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:43:54.121255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:43:54.121991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:43:54.122333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:43:54.122689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:43:54.124739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:43:54.212223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:43:54.213714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:43:54.225231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1263/2526] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:43:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:43:54.430597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:43:54.442478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba4df9-2a30-42b7-b97f-d94b8379f89e service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:43:54.447181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2382401-96a5-49b4-9cd2-820924439602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:43:54.448036 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2382401-96a5-49b4-9cd2-820924439602 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:43:54.448420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2382401-96a5-49b4-9cd2-820924439602 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:43:54.448795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2382401-96a5-49b4-9cd2-820924439602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:43:54.543164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2382401-96a5-49b4-9cd2-820924439602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:43:54.545391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2382401-96a5-49b4-9cd2-820924439602 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:43:54.557474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1264/2527] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:43:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:44:04.219664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=58 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:44:04.551889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=28 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:45:05.944478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:45:05.966200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3900da-991c-407c-8c87-522534174be7 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:45:05.973598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:45:05.974356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:45:05.974828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:45:05.975306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:45:06.075721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:45:06.077623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:45:06.104428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1264/2528] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:45:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:45:06.286784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2382401-96a5-49b4-9cd2-820924439602 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:45:06.297709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2382401-96a5-49b4-9cd2-820924439602 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:45:06.302712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d41b0-3d30-42f6-b6d2-f8e6813e36a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:45:06.303626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d41b0-3d30-42f6-b6d2-f8e6813e36a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:45:06.304033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d41b0-3d30-42f6-b6d2-f8e6813e36a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:45:06.304406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d41b0-3d30-42f6-b6d2-f8e6813e36a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:45:06.395879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d41b0-3d30-42f6-b6d2-f8e6813e36a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:45:06.397466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d41b0-3d30-42f6-b6d2-f8e6813e36a7 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:45:06.409889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1265/2529] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:45:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:45:16.100785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=36 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:45:16.403901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=28 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:45:51.599094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:45:51.601672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:45:51.690188 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c26aa-2620-4530-afc2-926a6c0f34ac service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:45:51.694272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:45:51.695015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:45:51.695324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:45:51.695599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:45:51.697739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:45:51.712186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:45:51.713662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:45:51.726541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1265/2530] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:45:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:45:54.883192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d41b0-3d30-42f6-b6d2-f8e6813e36a7 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:45:54.899135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d41b0-3d30-42f6-b6d2-f8e6813e36a7 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:45:54.910819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3755b6-dfa2-4143-95d3-a4fb444a7257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:45:54.910819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3755b6-dfa2-4143-95d3-a4fb444a7257 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:45:54.910819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3755b6-dfa2-4143-95d3-a4fb444a7257 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:45:54.910819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3755b6-dfa2-4143-95d3-a4fb444a7257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:45:55.009411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3755b6-dfa2-4143-95d3-a4fb444a7257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:45:55.011547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3755b6-dfa2-4143-95d3-a4fb444a7257 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:45:55.049346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1266/2531] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:45:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:46:01.721155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=26 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:46:05.044432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=36 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:47:06.505496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:47:06.521083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d42f7-49f6-4dc7-a02f-32de381f6035 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:47:06.525583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:47:06.526347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:47:06.527319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:47:06.527319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:47:06.542283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:47:06.544935 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:47:06.561204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1266/2532] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:47:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:47:06.902264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3755b6-dfa2-4143-95d3-a4fb444a7257 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:47:06.953048 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3755b6-dfa2-4143-95d3-a4fb444a7257 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:47:06.960956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:47:06.961947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:47:06.962394 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:47:06.962837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:47:06.966285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:47:07.040289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:47:07.042345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:47:07.056691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1267/2533] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:47:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:47:16.553901 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:47:17.056348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=22 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:49:07.648397 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:49:07.651289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:49:07.703957 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94f3d1-a297-40a5-bae3-3262edb8cd43 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:49:07.710906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f96a99-89cc-4531-b80f-c1f7a19e31f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:49:07.712040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f96a99-89cc-4531-b80f-c1f7a19e31f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:49:07.712513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f96a99-89cc-4531-b80f-c1f7a19e31f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:49:07.713002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f96a99-89cc-4531-b80f-c1f7a19e31f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:49:07.759052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f96a99-89cc-4531-b80f-c1f7a19e31f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:49:07.761841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f96a99-89cc-4531-b80f-c1f7a19e31f8 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:49:07.817289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1267/2534] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:49:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:49:07.879991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:49:07.911052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb451421-15bc-4f1f-bdd2-d72ac7e728bf service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:49:07.915876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:49:07.916517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:49:07.916804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:49:07.917071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:49:07.919364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:49:07.949188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:49:07.950845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:49:07.961838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1268/2535] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:49:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:49:17.812088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=28 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:49:17.956674 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=16 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:49:56.250664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f96a99-89cc-4531-b80f-c1f7a19e31f8 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:49:56.264949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f96a99-89cc-4531-b80f-c1f7a19e31f8 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:49:56.269976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:49:56.270859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:49:56.272147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:49:56.272797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:49:56.273241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:49:56.284238 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46543cff-1f43-471c-9d16-382ff073a849 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:49:56.288230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:49:56.288838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:49:56.289115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:49:56.289385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:49:56.292133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:49:56.375952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:49:56.378705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:49:56.397556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1268/2536] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:49:56.411333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:49:56.414367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:49:56.432342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1269/2537] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:50:02.360541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:02.365004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:02.390022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca4f5-c454-43b7-86cb-23af279758d6 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:02.397147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:02.398160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:02.398635 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:02.399456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:02.403558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:02.434748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:02.437682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:02.459593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1269/2538] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:50:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:50:02.472551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:02.494292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff615b59-5861-4a9f-bc4a-a7dd5264f659 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:02.526438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:02.528021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:02.530715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:02.533198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:02.535201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:02.535838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:02.539416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:02.564609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93185b78-ba07-4fd9-9eff-8d36f4c76f0d service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:02.577474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:02.578584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:02.579078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:02.579594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] PATH_INFO: `/v3/users/09b76f14426c4241aa49812f06f35cd1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:02.580860 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:02.581877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b76f14426c4241aa49812f06f35cd1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:02.586461 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:02.586706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:02.621933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1270/2539] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:50:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:50:02.697691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b76f14426c4241aa49812f06f35cd1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:02.698889 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b76f14426c4241aa49812f06f35cd1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:02.722763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b76f14426c4241aa49812f06f35cd1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:02.723928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b76f14426c4241aa49812f06f35cd1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:02.751916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-44e0db68-a4f9-4108-8dea-835ffbeca352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b76f14426c4241aa49812f06f35cd1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:02.780963 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1270/2540] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:50:02 2022] DELETE /identity/v3/users/09b76f14426c4241aa49812f06f35cd1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:50:03.083480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:03.117617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04a109-059d-4548-afdf-7d0c62c5a902 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:03.239990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:03.242530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:03.242530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:03.242889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] PATH_INFO: `/v3/projects/f16c9f15d25f47a8a986290c237084d1` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:03.247508 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16c9f15d25f47a8a986290c237084d1)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:03.248742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:03.314433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16c9f15d25f47a8a986290c237084d1', 'request_id': 'req-1180c024-fa65-437e-b7c2-583aebcc1413'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:03.328133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16c9f15d25f47a8a986290c237084d1', 'request_id': 'req-1180c024-fa65-437e-b7c2-583aebcc1413'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:03.343031 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16c9f15d25f47a8a986290c237084d1', 'request_id': 'req-1180c024-fa65-437e-b7c2-583aebcc1413'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:03.368616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] The token cache is being invalidate because project f16c9f15d25f47a8a986290c237084d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:50:03.369907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:03.370143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1180c024-fa65-437e-b7c2-583aebcc1413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:03.373672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1271/2541] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:50:03 2022] DELETE /identity/v3/projects/f16c9f15d25f47a8a986290c237084d1 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:03.382604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37b08-9bb6-4d7c-983c-118687cb596a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:03.383560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37b08-9bb6-4d7c-983c-118687cb596a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:03.383560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37b08-9bb6-4d7c-983c-118687cb596a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:03.417274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-43b37b08-9bb6-4d7c-983c-118687cb596a None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:03.418765 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-43b37b08-9bb6-4d7c-983c-118687cb596a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:03.495321 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:03.531290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1271/2542] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:50:03.535644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0279f-4550-465e-98a0-b7838b57cea0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:03.536794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0279f-4550-465e-98a0-b7838b57cea0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:03.536794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0279f-4550-465e-98a0-b7838b57cea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:03.575635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-71f0279f-4550-465e-98a0-b7838b57cea0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:03.577687 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-71f0279f-4550-465e-98a0-b7838b57cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:03.589062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1272/2543] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:50:03.595818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b37b08-9bb6-4d7c-983c-118687cb596a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:03.608114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b37b08-9bb6-4d7c-983c-118687cb596a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:03.614438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:03.616848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:03.617423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:03.617798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:03.618624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:50:03.619181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:03.621322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:03.638443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1272/2544] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:50:03.645263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0279f-4550-465e-98a0-b7838b57cea0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:03.651652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:03.658615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0279f-4550-465e-98a0-b7838b57cea0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:03.666288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d5014-a895-4815-b04b-1f90b9c29ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:03.667786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d5014-a895-4815-b04b-1f90b9c29ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:03.668225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d5014-a895-4815-b04b-1f90b9c29ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:03.668715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d5014-a895-4815-b04b-1f90b9c29ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:03.670323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d5014-a895-4815-b04b-1f90b9c29ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:03.674783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d5014-a895-4815-b04b-1f90b9c29ee8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:03.721010 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1273/2545] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:50:03 2022] POST /identity/v3/projects => generated 405 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:03.722684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:03.734797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32b041-a0e1-4342-bc91-0e182c53c8c6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:03.822205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc7585-15c4-48a0-8198-0cf31336909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:03.822943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc7585-15c4-48a0-8198-0cf31336909e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:03.823342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc7585-15c4-48a0-8198-0cf31336909e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:03.823680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc7585-15c4-48a0-8198-0cf31336909e admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:03.824814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc7585-15c4-48a0-8198-0cf31336909e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:03.827153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc7585-15c4-48a0-8198-0cf31336909e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:03.859313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1273/2546] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:50:03 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:03.864562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d5014-a895-4815-b04b-1f90b9c29ee8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:03.884062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d5014-a895-4815-b04b-1f90b9c29ee8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:03.895930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885e1f4-6b45-4e7a-beb9-c22e681eda75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:03.897020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e1f4-6b45-4e7a-beb9-c22e681eda75 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:03.897581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e1f4-6b45-4e7a-beb9-c22e681eda75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:03.899078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e1f4-6b45-4e7a-beb9-c22e681eda75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:03.899838 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885e1f4-6b45-4e7a-beb9-c22e681eda75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:03.903341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885e1f4-6b45-4e7a-beb9-c22e681eda75 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:03.923323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1274/2547] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:03.931406 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc7585-15c4-48a0-8198-0cf31336909e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:03.961459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc7585-15c4-48a0-8198-0cf31336909e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:03.971366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e72e1e-9d52-4c17-8351-38f45bde0270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:03.972926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e72e1e-9d52-4c17-8351-38f45bde0270 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:03.973525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e72e1e-9d52-4c17-8351-38f45bde0270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:03.975186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e72e1e-9d52-4c17-8351-38f45bde0270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:03.976770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e72e1e-9d52-4c17-8351-38f45bde0270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:03.981099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e72e1e-9d52-4c17-8351-38f45bde0270 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:04.011558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1274/2548] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:04.013724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885e1f4-6b45-4e7a-beb9-c22e681eda75 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:04.034266 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885e1f4-6b45-4e7a-beb9-c22e681eda75 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:04.045275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:04.046734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:04.047435 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:04.048089 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] PATH_INFO: `/v3/projects/e1acbb535b414e9aa19b5e1304f22be7/users/5ceb211822064b7c91a7b8076e554e54/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:04.083626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1acbb535b414e9aa19b5e1304f22be7, user_id=5ceb211822064b7c91a7b8076e554e54, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:04.089082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:04.122796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1275/2549] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:50:04 2022] PUT /identity/v3/projects/e1acbb535b414e9aa19b5e1304f22be7/users/5ceb211822064b7c91a7b8076e554e54/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:04.125638 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424d3e-97fd-4649-bf74-59a5baf86c46 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:04.126224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424d3e-97fd-4649-bf74-59a5baf86c46 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:04.126590 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424d3e-97fd-4649-bf74-59a5baf86c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:04.168521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6e424d3e-97fd-4649-bf74-59a5baf86c46 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:04.170868 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6e424d3e-97fd-4649-bf74-59a5baf86c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:04.268091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1275/2550] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:50:04.282986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:04.285114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:04.394234 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447887b-dffe-42c5-85e2-be9ee5d1a9ee admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:04.400608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf7d22-f805-4dca-9414-bf442614fabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:04.401664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7d22-f805-4dca-9414-bf442614fabb service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:04.402154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7d22-f805-4dca-9414-bf442614fabb service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:04.402634 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7d22-f805-4dca-9414-bf442614fabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:04.416346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf7d22-f805-4dca-9414-bf442614fabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:04.417876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf7d22-f805-4dca-9414-bf442614fabb service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:04.430359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1276/2551] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:50:05.528373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b3144-9035-48fa-b91c-73acf8e127b4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:05.529028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b3144-9035-48fa-b91c-73acf8e127b4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:05.529309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b3144-9035-48fa-b91c-73acf8e127b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:05.585591 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c58b3144-9035-48fa-b91c-73acf8e127b4 None None] MFA Rules not processed for user `5ceb211822064b7c91a7b8076e554e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:05.588099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c58b3144-9035-48fa-b91c-73acf8e127b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:05.686955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:05.739019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1276/2552] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:05 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:50:05.745989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf7d22-f805-4dca-9414-bf442614fabb service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:05.766323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf7d22-f805-4dca-9414-bf442614fabb service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:05.779694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a520293-db2b-434a-bb87-6f67bf2c2c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:05.780959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a520293-db2b-434a-bb87-6f67bf2c2c16 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:05.781544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a520293-db2b-434a-bb87-6f67bf2c2c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:05.782141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a520293-db2b-434a-bb87-6f67bf2c2c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:05.783998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a520293-db2b-434a-bb87-6f67bf2c2c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:05.792357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a520293-db2b-434a-bb87-6f67bf2c2c16 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:05.843171 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1277/2553] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:50:05 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:05.857959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b3144-9035-48fa-b91c-73acf8e127b4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:05.902949 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b3144-9035-48fa-b91c-73acf8e127b4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:06.051814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcdeaf-b62a-45a7-b592-4111402b814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:06.052531 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcdeaf-b62a-45a7-b592-4111402b814c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:06.053118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcdeaf-b62a-45a7-b592-4111402b814c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:06.053705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcdeaf-b62a-45a7-b592-4111402b814c admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:06.055042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcdeaf-b62a-45a7-b592-4111402b814c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:06.059061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcdeaf-b62a-45a7-b592-4111402b814c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:06.100304 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1277/2554] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:50:05 2022] POST /identity/v3/users => generated 309 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:06.105944 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a520293-db2b-434a-bb87-6f67bf2c2c16 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:06.128445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a520293-db2b-434a-bb87-6f67bf2c2c16 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:06.140667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3559cce-3667-4f17-ad3f-6fa807fe60e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:06.142198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3559cce-3667-4f17-ad3f-6fa807fe60e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:06.143020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3559cce-3667-4f17-ad3f-6fa807fe60e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:06.143677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3559cce-3667-4f17-ad3f-6fa807fe60e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:06.146784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3559cce-3667-4f17-ad3f-6fa807fe60e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:06.151382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3559cce-3667-4f17-ad3f-6fa807fe60e2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:06.185566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1278/2555] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:06.187741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcdeaf-b62a-45a7-b592-4111402b814c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:06.211559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcdeaf-b62a-45a7-b592-4111402b814c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:06.217422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7891d-7606-4a6f-9071-8575ae380216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:06.218137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7891d-7606-4a6f-9071-8575ae380216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:06.218417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7891d-7606-4a6f-9071-8575ae380216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:06.218902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7891d-7606-4a6f-9071-8575ae380216 admin admin] PATH_INFO: `/v3/projects/76361939f4584b18891a6b1d6a3c4e7a/users/165858ced8d148359e46e10929c203c3/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:06.257716 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7891d-7606-4a6f-9071-8575ae380216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76361939f4584b18891a6b1d6a3c4e7a, user_id=165858ced8d148359e46e10929c203c3, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:06.258852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7891d-7606-4a6f-9071-8575ae380216 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:06.288051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1278/2556] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:50:06 2022] PUT /identity/v3/projects/76361939f4584b18891a6b1d6a3c4e7a/users/165858ced8d148359e46e10929c203c3/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:06.292988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3559cce-3667-4f17-ad3f-6fa807fe60e2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:06.305043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3559cce-3667-4f17-ad3f-6fa807fe60e2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:06.403894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3a13-dc6c-4abb-ba16-e5ca1061ee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:06.404694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3a13-dc6c-4abb-ba16-e5ca1061ee67 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:06.404973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3a13-dc6c-4abb-ba16-e5ca1061ee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:06.405243 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3a13-dc6c-4abb-ba16-e5ca1061ee67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:06.405931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3a13-dc6c-4abb-ba16-e5ca1061ee67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:06.407858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3a13-dc6c-4abb-ba16-e5ca1061ee67 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:06.421120 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1279/2557] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:06.425738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7891d-7606-4a6f-9071-8575ae380216 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:06.451187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7891d-7606-4a6f-9071-8575ae380216 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:06.459280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:06.459280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:06.459789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:06.459789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] PATH_INFO: `/v3/projects/76361939f4584b18891a6b1d6a3c4e7a/users/165858ced8d148359e46e10929c203c3/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:06.462318 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76361939f4584b18891a6b1d6a3c4e7a, user_id=165858ced8d148359e46e10929c203c3, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:06.468071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:06.485140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1279/2558] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:50:06 2022] PUT /identity/v3/projects/76361939f4584b18891a6b1d6a3c4e7a/users/165858ced8d148359e46e10929c203c3/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:50:08.014599 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418adc8-ab18-48ba-a549-6a44862d2ebe None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:08.015564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418adc8-ab18-48ba-a549-6a44862d2ebe None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:08.015564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418adc8-ab18-48ba-a549-6a44862d2ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:08.063790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:08.072079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f418adc8-ab18-48ba-a549-6a44862d2ebe None None] MFA Rules not processed for user `165858ced8d148359e46e10929c203c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:08.073551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f418adc8-ab18-48ba-a549-6a44862d2ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:08.201337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:08.203392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:08.236019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1280/2559] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:50:08.320476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5ce98-b168-44f3-9e05-da41c3d1bf24 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:08.325948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:08.327097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:08.327560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:08.327988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] PATH_INFO: `/v3/users/d852fb6149224e0bbf3020bd68afb610` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:08.329913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d852fb6149224e0bbf3020bd68afb610)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:08.333345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:08.396965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:08.412308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd852fb6149224e0bbf3020bd68afb610'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:08.412308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd852fb6149224e0bbf3020bd68afb610'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:08.423312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd852fb6149224e0bbf3020bd68afb610'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:08.424678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd852fb6149224e0bbf3020bd68afb610'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:08.450276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd852fb6149224e0bbf3020bd68afb610'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:08.476907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1280/2560] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:50:08 2022] DELETE /identity/v3/users/d852fb6149224e0bbf3020bd68afb610 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:08.756009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418adc8-ab18-48ba-a549-6a44862d2ebe None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:08.777043 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418adc8-ab18-48ba-a549-6a44862d2ebe None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:08.903206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:08.904390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:08.904884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:08.905367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] PATH_INFO: `/v3/projects/a86e12cf23804019ad942a0a9fc245f0` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:08.907815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a86e12cf23804019ad942a0a9fc245f0)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:08.911751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:09.012146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a86e12cf23804019ad942a0a9fc245f0', 'request_id': 'req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.024250 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a86e12cf23804019ad942a0a9fc245f0', 'request_id': 'req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.046055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a86e12cf23804019ad942a0a9fc245f0', 'request_id': 'req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.083662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] The token cache is being invalidate because project a86e12cf23804019ad942a0a9fc245f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:50:09.084227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.085479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.088397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1281/2561] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:50:08 2022] DELETE /identity/v3/projects/a86e12cf23804019ad942a0a9fc245f0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:09.096326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:09.098383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:09.118048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecd7e9-6f45-4f2d-a078-3963493ce45d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:09.251420 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:09.252573 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:09.253025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:09.253758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] PATH_INFO: `/v3/users/01fe9242286a4bcda6465e24f471f8bc` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:09.256207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01fe9242286a4bcda6465e24f471f8bc)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:09.260034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:09.347933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01fe9242286a4bcda6465e24f471f8bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.349607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01fe9242286a4bcda6465e24f471f8bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.359747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01fe9242286a4bcda6465e24f471f8bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.360608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01fe9242286a4bcda6465e24f471f8bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.388867 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01fe9242286a4bcda6465e24f471f8bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:09.409634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1281/2562] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:50:09 2022] DELETE /identity/v3/users/01fe9242286a4bcda6465e24f471f8bc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:09.818601 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:09.830918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:09.864423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1910a-f8d0-4cb7-a311-c2879fedd1bd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:09.937927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:10.001880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:10.002726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:10.003200 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:10.003764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] PATH_INFO: `/v3/projects/600e1c00222740fb95fc38800d98a9ac` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:10.005984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=600e1c00222740fb95fc38800d98a9ac)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:10.010045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:10.105869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600e1c00222740fb95fc38800d98a9ac', 'request_id': 'req-e6f11d5d-442c-4013-ab1d-c482fa3020d0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.114018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:10.116592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:10.120106 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600e1c00222740fb95fc38800d98a9ac', 'request_id': 'req-e6f11d5d-442c-4013-ab1d-c482fa3020d0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.133895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600e1c00222740fb95fc38800d98a9ac', 'request_id': 'req-e6f11d5d-442c-4013-ab1d-c482fa3020d0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.147503 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb73c8-dffd-4df4-b40e-613db890be7c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:10.167650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] The token cache is being invalidate because project 600e1c00222740fb95fc38800d98a9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:50:10.168131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.169281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.179453 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1282/2563] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:50:09 2022] DELETE /identity/v3/projects/600e1c00222740fb95fc38800d98a9ac => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:50:10.218298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:10.277509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:10.278613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:10.279097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:10.280215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] PATH_INFO: `/v3/users/5ceb211822064b7c91a7b8076e554e54` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:10.283425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ceb211822064b7c91a7b8076e554e54)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:10.289749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:10.379033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ceb211822064b7c91a7b8076e554e54'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.379884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ceb211822064b7c91a7b8076e554e54'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.397092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ceb211822064b7c91a7b8076e554e54'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.398109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ceb211822064b7c91a7b8076e554e54'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.423340 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ceb211822064b7c91a7b8076e554e54'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.448506 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1282/2564] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:50:10 2022] DELETE /identity/v3/users/5ceb211822064b7c91a7b8076e554e54 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:50:10.745616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:10.747595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:10.764281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f11d5d-442c-4013-ab1d-c482fa3020d0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:10.870077 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:10.871269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:10.871743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:10.872197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] PATH_INFO: `/v3/projects/e1acbb535b414e9aa19b5e1304f22be7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:10.874384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1acbb535b414e9aa19b5e1304f22be7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:10.878026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:10.957903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1acbb535b414e9aa19b5e1304f22be7', 'request_id': 'req-f083c130-726d-4286-b757-4f8c2d2cd320'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.967273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1acbb535b414e9aa19b5e1304f22be7', 'request_id': 'req-f083c130-726d-4286-b757-4f8c2d2cd320'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:10.971215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:10.981292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1acbb535b414e9aa19b5e1304f22be7', 'request_id': 'req-f083c130-726d-4286-b757-4f8c2d2cd320'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:11.006440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] The token cache is being invalidate because project e1acbb535b414e9aa19b5e1304f22be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:50:11.006901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:11.007766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:11.009491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1283/2565] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:50:10 2022] DELETE /identity/v3/projects/e1acbb535b414e9aa19b5e1304f22be7 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:11.014673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:11.016970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:11.034162 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0495c-fe13-4128-8f74-e9179dfc3115 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:11.295953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:11.297013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:11.297384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:11.297787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] PATH_INFO: `/v3/users/165858ced8d148359e46e10929c203c3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:11.299456 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=165858ced8d148359e46e10929c203c3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:11.302289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:11.380547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '165858ced8d148359e46e10929c203c3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:11.381409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '165858ced8d148359e46e10929c203c3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:11.395073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '165858ced8d148359e46e10929c203c3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:11.398435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '165858ced8d148359e46e10929c203c3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:11.401205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:11.422711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-21b7c137-d4c7-4606-b542-e7ad5284eb8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '165858ced8d148359e46e10929c203c3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:11.438147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1283/2566] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:50:11 2022] DELETE /identity/v3/users/165858ced8d148359e46e10929c203c3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:11.732282 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:11.734900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:11.766425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c130-726d-4286-b757-4f8c2d2cd320 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:11.896291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:11.897314 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:11.897943 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:11.898411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] PATH_INFO: `/v3/projects/76361939f4584b18891a6b1d6a3c4e7a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:11.900546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76361939f4584b18891a6b1d6a3c4e7a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:11.904227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:11.986560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76361939f4584b18891a6b1d6a3c4e7a', 'request_id': 'req-ec929f13-aea1-4141-99e1-ff5fe495b7f9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:12.001982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76361939f4584b18891a6b1d6a3c4e7a', 'request_id': 'req-ec929f13-aea1-4141-99e1-ff5fe495b7f9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:12.014891 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76361939f4584b18891a6b1d6a3c4e7a', 'request_id': 'req-ec929f13-aea1-4141-99e1-ff5fe495b7f9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:12.046874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] The token cache is being invalidate because project 76361939f4584b18891a6b1d6a3c4e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:50:12.047431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:12.048576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ec929f13-aea1-4141-99e1-ff5fe495b7f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:50:12.051472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1284/2567] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:50:11 2022] DELETE /identity/v3/projects/76361939f4584b18891a6b1d6a3c4e7a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:12.059944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbfd03-8f0b-4c58-8992-d3d591a22ebb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.060440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbfd03-8f0b-4c58-8992-d3d591a22ebb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.060884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbfd03-8f0b-4c58-8992-d3d591a22ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.087774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5bbbfd03-8f0b-4c58-8992-d3d591a22ebb None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:12.089755 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5bbbfd03-8f0b-4c58-8992-d3d591a22ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:12.193554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1284/2568] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:50:12.202888 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbec1af-1a9c-40bf-9932-2850b15bcf82 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.211487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbec1af-1a9c-40bf-9932-2850b15bcf82 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.211487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbec1af-1a9c-40bf-9932-2850b15bcf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.230840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-adbec1af-1a9c-40bf-9932-2850b15bcf82 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:12.232299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-adbec1af-1a9c-40bf-9932-2850b15bcf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:12.247334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1285/2569] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:50:12.254105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbfd03-8f0b-4c58-8992-d3d591a22ebb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:12.267609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbfd03-8f0b-4c58-8992-d3d591a22ebb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:12.275754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:12.276677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.277137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.277523 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.278556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:50:12.279026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:12.281858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:12.298519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1285/2570] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:50:12.304947 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbec1af-1a9c-40bf-9932-2850b15bcf82 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:12.322650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbec1af-1a9c-40bf-9932-2850b15bcf82 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:12.331653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657a58d-e50c-4169-904f-3fc8c947d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:12.332828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657a58d-e50c-4169-904f-3fc8c947d75f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.333395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657a58d-e50c-4169-904f-3fc8c947d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.333688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657a58d-e50c-4169-904f-3fc8c947d75f admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.334849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657a58d-e50c-4169-904f-3fc8c947d75f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:12.338583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657a58d-e50c-4169-904f-3fc8c947d75f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:12.390764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1286/2571] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:50:12 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:12.396084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:12.412376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f215af-bc16-4e51-b018-e3d8dbc4fc1b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:12.518121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670482ec-cca7-4aa3-9e38-8fe01ececbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:12.519018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670482ec-cca7-4aa3-9e38-8fe01ececbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.519469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670482ec-cca7-4aa3-9e38-8fe01ececbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.519902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670482ec-cca7-4aa3-9e38-8fe01ececbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.521090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670482ec-cca7-4aa3-9e38-8fe01ececbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:12.524264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670482ec-cca7-4aa3-9e38-8fe01ececbb5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:12.560021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1286/2572] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:50:12 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:12.564961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657a58d-e50c-4169-904f-3fc8c947d75f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:12.576886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657a58d-e50c-4169-904f-3fc8c947d75f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:12.584336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ef14c-2471-4830-9487-4193c41484d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:12.585385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ef14c-2471-4830-9487-4193c41484d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.585827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ef14c-2471-4830-9487-4193c41484d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.586261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ef14c-2471-4830-9487-4193c41484d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.587022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ef14c-2471-4830-9487-4193c41484d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:12.588985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ef14c-2471-4830-9487-4193c41484d9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:12.605578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1287/2573] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:12.610845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670482ec-cca7-4aa3-9e38-8fe01ececbb5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:12.623547 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670482ec-cca7-4aa3-9e38-8fe01ececbb5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:12.628658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71cdac-cd41-4b32-86dc-169ecb966850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:12.630129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71cdac-cd41-4b32-86dc-169ecb966850 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.630416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71cdac-cd41-4b32-86dc-169ecb966850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.630688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71cdac-cd41-4b32-86dc-169ecb966850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.631398 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71cdac-cd41-4b32-86dc-169ecb966850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:12.633037 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71cdac-cd41-4b32-86dc-169ecb966850 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:12.645993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1287/2574] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:12.651661 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ef14c-2471-4830-9487-4193c41484d9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:12.670535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ef14c-2471-4830-9487-4193c41484d9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:12.679016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f088b-f9a6-430e-a8b5-9b38d31dfc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:12.680265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f088b-f9a6-430e-a8b5-9b38d31dfc2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.680769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f088b-f9a6-430e-a8b5-9b38d31dfc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.681203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f088b-f9a6-430e-a8b5-9b38d31dfc2d admin admin] PATH_INFO: `/v3/projects/a0b6cbf58da740449ec001a599406683/users/de620b43e967425cb83d8cbfa9e0f932/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.709973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f088b-f9a6-430e-a8b5-9b38d31dfc2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b6cbf58da740449ec001a599406683, user_id=de620b43e967425cb83d8cbfa9e0f932, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:12.715329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f088b-f9a6-430e-a8b5-9b38d31dfc2d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:12.737517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1288/2575] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:50:12 2022] PUT /identity/v3/projects/a0b6cbf58da740449ec001a599406683/users/de620b43e967425cb83d8cbfa9e0f932/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:12.754948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e2442-f1bb-496b-ba2b-d524c4fd9c1a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:12.755429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e2442-f1bb-496b-ba2b-d524c4fd9c1a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:12.755888 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e2442-f1bb-496b-ba2b-d524c4fd9c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:12.802728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b78e2442-f1bb-496b-ba2b-d524c4fd9c1a None None] MFA Rules not processed for user `de620b43e967425cb83d8cbfa9e0f932`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:12.803819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b78e2442-f1bb-496b-ba2b-d524c4fd9c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:12.852306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:12.933691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1288/2576] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:50:12.941164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f088b-f9a6-430e-a8b5-9b38d31dfc2d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:12.975620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:13.069257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f088b-f9a6-430e-a8b5-9b38d31dfc2d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:13.077523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a845b-215a-4418-bde7-7a2d64d374f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:13.078548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a845b-215a-4418-bde7-7a2d64d374f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:13.079035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a845b-215a-4418-bde7-7a2d64d374f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:13.079562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a845b-215a-4418-bde7-7a2d64d374f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:13.080708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a845b-215a-4418-bde7-7a2d64d374f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:13.084432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a845b-215a-4418-bde7-7a2d64d374f6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:13.129590 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1289/2577] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:50:12 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:13.134284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e2442-f1bb-496b-ba2b-d524c4fd9c1a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:13.150100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e2442-f1bb-496b-ba2b-d524c4fd9c1a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:13.226137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2347ff-08cc-4bb3-815b-711d95114a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:13.226790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2347ff-08cc-4bb3-815b-711d95114a98 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:13.227051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2347ff-08cc-4bb3-815b-711d95114a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:13.227455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2347ff-08cc-4bb3-815b-711d95114a98 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:13.228054 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2347ff-08cc-4bb3-815b-711d95114a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:13.230071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2347ff-08cc-4bb3-815b-711d95114a98 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:13.258955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1289/2578] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:50:13 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:13.264251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a845b-215a-4418-bde7-7a2d64d374f6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:13.295274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a845b-215a-4418-bde7-7a2d64d374f6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:13.304544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d348ce7-4af5-4f86-b8ae-ba5be0ec4f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:13.305531 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d348ce7-4af5-4f86-b8ae-ba5be0ec4f4b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:13.305988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d348ce7-4af5-4f86-b8ae-ba5be0ec4f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:13.306421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d348ce7-4af5-4f86-b8ae-ba5be0ec4f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:13.307635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d348ce7-4af5-4f86-b8ae-ba5be0ec4f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:13.310644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d348ce7-4af5-4f86-b8ae-ba5be0ec4f4b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:13.337604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1290/2579] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:13.337906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2347ff-08cc-4bb3-815b-711d95114a98 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:13.350693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2347ff-08cc-4bb3-815b-711d95114a98 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:13.358382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb988cf-a477-49b4-a243-8b87fe8a53be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:13.359464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb988cf-a477-49b4-a243-8b87fe8a53be admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:13.359945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb988cf-a477-49b4-a243-8b87fe8a53be admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:13.360417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb988cf-a477-49b4-a243-8b87fe8a53be admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:13.361458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb988cf-a477-49b4-a243-8b87fe8a53be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:13.363734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb988cf-a477-49b4-a243-8b87fe8a53be admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:13.377261 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1290/2580] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:13.384797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d348ce7-4af5-4f86-b8ae-ba5be0ec4f4b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:13.402805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d348ce7-4af5-4f86-b8ae-ba5be0ec4f4b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:13.410652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af48363-104b-4861-923f-cddb7392ed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:13.411869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af48363-104b-4861-923f-cddb7392ed0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:13.412360 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af48363-104b-4861-923f-cddb7392ed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:13.412853 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af48363-104b-4861-923f-cddb7392ed0e admin admin] PATH_INFO: `/v3/projects/f655224fff6c44b8a8277fcea0d1657d/users/cdbdcb2584694d7aa6118d05a6651d42/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:13.441889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af48363-104b-4861-923f-cddb7392ed0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f655224fff6c44b8a8277fcea0d1657d, user_id=cdbdcb2584694d7aa6118d05a6651d42, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:13.447896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af48363-104b-4861-923f-cddb7392ed0e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:13.472473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1291/2581] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:50:13 2022] PUT /identity/v3/projects/f655224fff6c44b8a8277fcea0d1657d/users/cdbdcb2584694d7aa6118d05a6651d42/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:13.493027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c2aa1-c622-46e9-90bd-3bce2c7f7705 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:13.493387 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c2aa1-c622-46e9-90bd-3bce2c7f7705 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:13.493834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c2aa1-c622-46e9-90bd-3bce2c7f7705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:13.545415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2c4c2aa1-c622-46e9-90bd-3bce2c7f7705 None None] MFA Rules not processed for user `cdbdcb2584694d7aa6118d05a6651d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:13.547074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2c4c2aa1-c622-46e9-90bd-3bce2c7f7705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:13.660031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1291/2582] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:50:13.668436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af48363-104b-4861-923f-cddb7392ed0e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:13.759808 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af48363-104b-4861-923f-cddb7392ed0e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:13.765182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e6969-05e3-4efc-be84-7ce2e6b3fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:13.765948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e6969-05e3-4efc-be84-7ce2e6b3fb77 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:13.766316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e6969-05e3-4efc-be84-7ce2e6b3fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:13.766685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e6969-05e3-4efc-be84-7ce2e6b3fb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:13.767520 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e6969-05e3-4efc-be84-7ce2e6b3fb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:13.769595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e6969-05e3-4efc-be84-7ce2e6b3fb77 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:13.812084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1292/2583] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:50:13 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:13.818520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c2aa1-c622-46e9-90bd-3bce2c7f7705 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:13.838346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c2aa1-c622-46e9-90bd-3bce2c7f7705 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:13.907939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:13.963535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc188de-0a14-4c0c-a204-d23e4e7e42eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:13.964485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc188de-0a14-4c0c-a204-d23e4e7e42eb admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:13.964923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc188de-0a14-4c0c-a204-d23e4e7e42eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:13.965341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc188de-0a14-4c0c-a204-d23e4e7e42eb admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:13.966356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc188de-0a14-4c0c-a204-d23e4e7e42eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:13.968877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc188de-0a14-4c0c-a204-d23e4e7e42eb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:14.000237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1292/2584] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:50:13 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:50:14.006186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e6969-05e3-4efc-be84-7ce2e6b3fb77 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:14.023565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e6969-05e3-4efc-be84-7ce2e6b3fb77 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:14.028620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43934e-d5d9-47ac-826d-5f4128f0c649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:14.029390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43934e-d5d9-47ac-826d-5f4128f0c649 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:14.029779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43934e-d5d9-47ac-826d-5f4128f0c649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:14.030143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43934e-d5d9-47ac-826d-5f4128f0c649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:14.030898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43934e-d5d9-47ac-826d-5f4128f0c649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:14.032809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43934e-d5d9-47ac-826d-5f4128f0c649 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:14.048985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1293/2585] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:50:14.055962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc188de-0a14-4c0c-a204-d23e4e7e42eb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:14.074137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc188de-0a14-4c0c-a204-d23e4e7e42eb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:14.082255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696b25d-210a-4022-b6a5-81f740bf07ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:14.083119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696b25d-210a-4022-b6a5-81f740bf07ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:14.083785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696b25d-210a-4022-b6a5-81f740bf07ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:14.084144 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696b25d-210a-4022-b6a5-81f740bf07ee admin admin] PATH_INFO: `/v3/projects/151a7b42e4874d3e862dbb278abf3171/users/3207e35a6e924b9380a1ea4412ab5998/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:14.108566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696b25d-210a-4022-b6a5-81f740bf07ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=151a7b42e4874d3e862dbb278abf3171, user_id=3207e35a6e924b9380a1ea4412ab5998, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:14.111268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696b25d-210a-4022-b6a5-81f740bf07ee admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:14.137769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1293/2586] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:50:14 2022] PUT /identity/v3/projects/151a7b42e4874d3e862dbb278abf3171/users/3207e35a6e924b9380a1ea4412ab5998/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:14.143388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43934e-d5d9-47ac-826d-5f4128f0c649 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:14.162447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43934e-d5d9-47ac-826d-5f4128f0c649 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:14.224502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:14.257464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796359-e39d-43b9-a1ac-6bab81f97e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:14.258308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796359-e39d-43b9-a1ac-6bab81f97e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:14.258567 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796359-e39d-43b9-a1ac-6bab81f97e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:14.258837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796359-e39d-43b9-a1ac-6bab81f97e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:14.259624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796359-e39d-43b9-a1ac-6bab81f97e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:14.261333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796359-e39d-43b9-a1ac-6bab81f97e20 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:14.274605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1294/2587] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:50:14.279140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696b25d-210a-4022-b6a5-81f740bf07ee admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:14.295703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696b25d-210a-4022-b6a5-81f740bf07ee admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:14.303523 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:14.304622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:14.305066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:14.305483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] PATH_INFO: `/v3/projects/151a7b42e4874d3e862dbb278abf3171/users/3207e35a6e924b9380a1ea4412ab5998/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:14.308823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=151a7b42e4874d3e862dbb278abf3171, user_id=3207e35a6e924b9380a1ea4412ab5998, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:14.313815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:14.332776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1294/2588] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:50:14 2022] PUT /identity/v3/projects/151a7b42e4874d3e862dbb278abf3171/users/3207e35a6e924b9380a1ea4412ab5998/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:50:14.352885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074a583-399f-46fa-bc36-6cffaa135bb3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:14.352885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074a583-399f-46fa-bc36-6cffaa135bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:14.352885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074a583-399f-46fa-bc36-6cffaa135bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:14.391154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3074a583-399f-46fa-bc36-6cffaa135bb3 None None] MFA Rules not processed for user `3207e35a6e924b9380a1ea4412ab5998`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:50:14.396260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3074a583-399f-46fa-bc36-6cffaa135bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:14.507638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1295/2589] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:50:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:50:24.325657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=28,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:24.502240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=45 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:50:56.539147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:50:56.541309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:56.650863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eebdf8-49fd-4bf0-966d-5a92e4dec7f5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:50:56.657369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:50:56.658421 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:50:56.658915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:50:56.659466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:50:56.662540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:50:56.686943 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:50:56.691486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:50:56.708933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1295/2590] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:50:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:51:06.701566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=30 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:51:08.309272 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074a583-399f-46fa-bc36-6cffaa135bb3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:51:08.311642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3074a583-399f-46fa-bc36-6cffaa135bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:51:08.410942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074a583-399f-46fa-bc36-6cffaa135bb3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:51:08.417692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:51:08.418751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:51:08.419219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:51:08.419725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:51:08.422739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:51:08.536365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:51:08.538007 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:51:08.550527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1296/2591] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:51:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:51:08.771618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:51:08.791542 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f7479-52d1-466d-b4c7-899bd9a6a09a service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:51:08.797362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:51:08.798092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:51:08.798489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:51:08.798871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:51:08.801962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:51:08.897172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:51:08.898999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:51:08.919953 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1296/2592] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:51:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:51:08.985928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:51:09.001494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff63af-869f-4c48-9c27-cd3667b8e7c1 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:51:09.008266 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:51:09.009309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:51:09.009773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:51:09.010265 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:51:09.027898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:51:09.030784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:51:09.050366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1297/2593] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:51:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:51:09.233309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:51:09.245602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24512-45a1-40ab-abb1-43c4e959f2ab service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:51:09.249801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:51:09.250427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:51:09.250721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:51:09.251086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:51:09.252991 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:51:09.372200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:51:09.373743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:51:09.398234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1297/2594] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:51:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:51:19.041668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=59 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:51:19.395524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=74 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:51:56.111011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:51:56.113429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:51:56.132855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a2b0d-ded5-478c-97bb-5bbe8391778e service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:51:56.139404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:51:56.140385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:51:56.140842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:51:56.141311 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:51:56.234345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:51:56.236078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:51:56.248211 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1298/2595] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:51:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:52:06.242806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=28 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:52:13.868682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:52:13.870341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:52:13.968783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a5062-7d3a-4f2e-83bd-3d7ffba1d6c9 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:52:13.975137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46face-377d-4916-8541-2eba625e498e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:52:13.975813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46face-377d-4916-8541-2eba625e498e service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:52:13.976093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46face-377d-4916-8541-2eba625e498e service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:52:13.976366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46face-377d-4916-8541-2eba625e498e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:52:13.978351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0a46face-377d-4916-8541-2eba625e498e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:52:13.992367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46face-377d-4916-8541-2eba625e498e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:52:13.994047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46face-377d-4916-8541-2eba625e498e service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:52:14.005746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1298/2596] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:52:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:52:24.000811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=32 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:53.467663 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:53.476646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:53.649886 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ee6f0-97db-42cb-92c3-665fb70ed0a3 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:53.657329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38805bc3-3838-44fb-adbd-df3a4be0e982 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:53.658068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38805bc3-3838-44fb-adbd-df3a4be0e982 service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:53.659026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38805bc3-3838-44fb-adbd-df3a4be0e982 service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:53.662315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38805bc3-3838-44fb-adbd-df3a4be0e982 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:53.664841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-38805bc3-3838-44fb-adbd-df3a4be0e982 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-38805bc3-3838-44fb-adbd-df3a4be0e982 service glance] Could not find project: a86e12cf23804019ad942a0a9fc245f0.: keystone.exception.ProjectNotFound: Could not find project: a86e12cf23804019ad942a0a9fc245f0. Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 31 12:53:53.702363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:53:53.704522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 12:53:53.707408 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 12:53:53.709698 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 12:53:53.713271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a86e12cf23804019ad942a0a9fc245f0. Aug 31 12:53:53.713271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: ERROR keystone.server.flask.application Aug 31 12:53:53.713271 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1299/2597] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:53 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 31 12:53:54.068566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46face-377d-4916-8541-2eba625e498e service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:54.092332 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46face-377d-4916-8541-2eba625e498e service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:54.106656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc721c3-7b2d-4ac2-8c71-cf3aa640e3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:54.107784 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc721c3-7b2d-4ac2-8c71-cf3aa640e3db service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:54.108501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc721c3-7b2d-4ac2-8c71-cf3aa640e3db service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:54.109176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc721c3-7b2d-4ac2-8c71-cf3aa640e3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:54.121448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8bc721c3-7b2d-4ac2-8c71-cf3aa640e3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:54.192462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38805bc3-3838-44fb-adbd-df3a4be0e982 service glance] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:54.312155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc721c3-7b2d-4ac2-8c71-cf3aa640e3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:54.314841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc721c3-7b2d-4ac2-8c71-cf3aa640e3db service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:54.320251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38805bc3-3838-44fb-adbd-df3a4be0e982 service glance] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:54.337625 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:54.338564 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:54.338841 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:54.339212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] PATH_INFO: `/v3/users/de620b43e967425cb83d8cbfa9e0f932` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:54.341023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] RBAC: Authorizing `identity:delete_user(user_id=de620b43e967425cb83d8cbfa9e0f932)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:54.344908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:54.371984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1299/2598] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:53:54.449742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de620b43e967425cb83d8cbfa9e0f932'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:54.450652 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de620b43e967425cb83d8cbfa9e0f932'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:54.474862 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de620b43e967425cb83d8cbfa9e0f932'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:54.486789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:54.490985 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de620b43e967425cb83d8cbfa9e0f932'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:54.503930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de620b43e967425cb83d8cbfa9e0f932'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:54.513885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1300/2599] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:53:54 2022] DELETE /identity/v3/users/de620b43e967425cb83d8cbfa9e0f932 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:53:54.519034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962997f0-6978-44eb-8a71-57488ef91af0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:54.519357 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962997f0-6978-44eb-8a71-57488ef91af0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:54.519628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962997f0-6978-44eb-8a71-57488ef91af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:54.596639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-962997f0-6978-44eb-8a71-57488ef91af0 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:53:54.598344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-962997f0-6978-44eb-8a71-57488ef91af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:54.698795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1300/2600] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:53:54.713607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:54.715420 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:54.873351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864e0b5-7677-41f2-9595-5694a59addad admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:54.882725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:54.884649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:54.885528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:54.886482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:54.934648 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:54.938847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:54.985276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1301/2601] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:53:55.031887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962997f0-6978-44eb-8a71-57488ef91af0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:55.047087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962997f0-6978-44eb-8a71-57488ef91af0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:55.053577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:55.054761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:55.055106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:55.055484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:55.058129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:55.079330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:55.085584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:55.107834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1301/2602] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:53:55.248143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:55.249655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:55.275915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dad87-d826-4e83-bf8c-f6698705f931 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:55.284450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7e8af-b91c-495f-9bd6-1b9e790be6ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:55.285939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e8af-b91c-495f-9bd6-1b9e790be6ae service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:55.285939 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e8af-b91c-495f-9bd6-1b9e790be6ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:55.287198 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e8af-b91c-495f-9bd6-1b9e790be6ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:55.290172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3bf7e8af-b91c-495f-9bd6-1b9e790be6ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:55.332366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7e8af-b91c-495f-9bd6-1b9e790be6ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:55.334677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7e8af-b91c-495f-9bd6-1b9e790be6ae service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:55.357479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1302/2603] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:53:55.759683 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:55.808749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a0ca6-167b-4318-a415-117091c6fd77 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:55.830887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9089c-5453-45e7-be79-bd493208af84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:55.834931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9089c-5453-45e7-be79-bd493208af84 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:55.836366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9089c-5453-45e7-be79-bd493208af84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:55.837124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9089c-5453-45e7-be79-bd493208af84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:55.842548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-01a9089c-5453-45e7-be79-bd493208af84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:55.922457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:56.010792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9089c-5453-45e7-be79-bd493208af84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:56.013228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9089c-5453-45e7-be79-bd493208af84 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:56.029898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1302/2604] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:53:56.135555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae97343-53d0-4a91-91bc-b8634daeb054 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:56.136122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae97343-53d0-4a91-91bc-b8634daeb054 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:56.136122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae97343-53d0-4a91-91bc-b8634daeb054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:56.284647 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6ae97343-53d0-4a91-91bc-b8634daeb054 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:53:56.286377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6ae97343-53d0-4a91-91bc-b8634daeb054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:56.300014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1303/2605] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:53:56.306896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3417bc-67c9-46c6-8bc8-979656b38f53 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:56.307608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3417bc-67c9-46c6-8bc8-979656b38f53 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:56.307608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3417bc-67c9-46c6-8bc8-979656b38f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:56.331490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cc3417bc-67c9-46c6-8bc8-979656b38f53 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:53:56.333137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cc3417bc-67c9-46c6-8bc8-979656b38f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:56.345555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1303/2606] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:53:56.348352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae97343-53d0-4a91-91bc-b8634daeb054 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:56.400254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae97343-53d0-4a91-91bc-b8634daeb054 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:56.409593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:56.411133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:56.411748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:56.412487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:56.413736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:53:56.414289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:56.417494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:56.434075 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3417bc-67c9-46c6-8bc8-979656b38f53 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:56.435923 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cc3417bc-67c9-46c6-8bc8-979656b38f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:56.451906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1304/2607] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:53:56.454032 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:56.456007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3417bc-67c9-46c6-8bc8-979656b38f53 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:56.462726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:56.463931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:56.464510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:56.465269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] PATH_INFO: `/v3/projects/a0b6cbf58da740449ec001a599406683` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:56.466971 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b6cbf58da740449ec001a599406683)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:56.469411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:56.479269 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271183e-c1b2-4837-935c-8862242d636f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:56.495622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3a801-9a6b-405e-a25f-a7c88d68f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:56.497313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3a801-9a6b-405e-a25f-a7c88d68f8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:56.498874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3a801-9a6b-405e-a25f-a7c88d68f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:56.500050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3a801-9a6b-405e-a25f-a7c88d68f8ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:56.501517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3a801-9a6b-405e-a25f-a7c88d68f8ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:56.505924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3a801-9a6b-405e-a25f-a7c88d68f8ee admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:56.580840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b6cbf58da740449ec001a599406683', 'request_id': 'req-dcff447c-2fe7-4c66-8f40-415c7f594a78'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:56.592736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1305/2608] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:53:56 2022] POST /identity/v3/projects => generated 399 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:53:56.598249 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3a801-9a6b-405e-a25f-a7c88d68f8ee admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:56.601842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b6cbf58da740449ec001a599406683', 'request_id': 'req-dcff447c-2fe7-4c66-8f40-415c7f594a78'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:56.632660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b6cbf58da740449ec001a599406683', 'request_id': 'req-dcff447c-2fe7-4c66-8f40-415c7f594a78'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:56.666615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] The token cache is being invalidate because project a0b6cbf58da740449ec001a599406683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:53:56.667069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:56.668449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:56.672516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1304/2609] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:53:56 2022] DELETE /identity/v3/projects/a0b6cbf58da740449ec001a599406683 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:53:56.676785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:56.679462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:56.811473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:56.879322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3a801-9a6b-405e-a25f-a7c88d68f8ee admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:56.894868 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:56.900022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:56.901362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:56.901902 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:56.903154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:56.907102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:56.963153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1306/2610] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:53:56 2022] POST /identity/v3/users => generated 300 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:53:56.964725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:56.967792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:57.009218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b0068-48bf-4292-ace3-d3bb5a8b7ec6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:57.016835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff447c-2fe7-4c66-8f40-415c7f594a78 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:57.035489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:57.039798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:57.040342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:57.040778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] PATH_INFO: `/v3/users/cdbdcb2584694d7aa6118d05a6651d42` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:57.056481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdbdcb2584694d7aa6118d05a6651d42)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:57.059593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:57.152926 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:57.232019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdbdcb2584694d7aa6118d05a6651d42'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:57.234102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdbdcb2584694d7aa6118d05a6651d42'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:57.286646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdbdcb2584694d7aa6118d05a6651d42'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:57.329864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdbdcb2584694d7aa6118d05a6651d42'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:57.330165 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:57.331464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:57.331952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:57.332395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:57.341409 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:57.351574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdbdcb2584694d7aa6118d05a6651d42'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:57.360877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1307/2611] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:53:56 2022] DELETE /identity/v3/users/cdbdcb2584694d7aa6118d05a6651d42 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:53:57.361563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:57.368732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:57.370366 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:57.382184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:57.384478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1305/2612] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:53:57.397720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:57.401462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:57.435179 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec736a-4eba-41d3-8f03-3c2f7c685217 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:57.582947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:57.583484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:57.583780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:57.584072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:57.586770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:57.603597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:57.605445 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:57.620920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1306/2613] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:53:57.690451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9200478-a3b3-487d-9238-189d835b9b8d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:57.718722 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47965936-13bc-44cd-92c7-8a13aa89b448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:57.720240 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47965936-13bc-44cd-92c7-8a13aa89b448 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:57.723909 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47965936-13bc-44cd-92c7-8a13aa89b448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:57.724544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47965936-13bc-44cd-92c7-8a13aa89b448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:57.728361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47965936-13bc-44cd-92c7-8a13aa89b448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:57.738828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47965936-13bc-44cd-92c7-8a13aa89b448 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:57.791707 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1308/2614] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:53:57.792780 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:57.817682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d224d89-21e9-4c85-88c4-0c1334514e91 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:57.839793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1420b41-e6d1-42be-9916-272c6f0244a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:57.839793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420b41-e6d1-42be-9916-272c6f0244a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:57.839793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420b41-e6d1-42be-9916-272c6f0244a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:57.840477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420b41-e6d1-42be-9916-272c6f0244a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:57.841112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1420b41-e6d1-42be-9916-272c6f0244a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:57.843058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1420b41-e6d1-42be-9916-272c6f0244a8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:57.867126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1307/2615] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:53:57.874732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47965936-13bc-44cd-92c7-8a13aa89b448 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:57.893521 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47965936-13bc-44cd-92c7-8a13aa89b448 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:57.905448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:57.908156 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:57.911305 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:57.911726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] PATH_INFO: `/v3/projects/35da2362b7d243feb77b2dda2f490b0c/users/2b2065bb4a1c4ba7b435d1ce53eca71e/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:57.936781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35da2362b7d243feb77b2dda2f490b0c, user_id=2b2065bb4a1c4ba7b435d1ce53eca71e, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:57.956236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:57.996776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1309/2616] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:53:57 2022] PUT /identity/v3/projects/35da2362b7d243feb77b2dda2f490b0c/users/2b2065bb4a1c4ba7b435d1ce53eca71e/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:53:58.020869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8547dd-17f3-46f2-b063-4f249bac6508 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:58.021163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8547dd-17f3-46f2-b063-4f249bac6508 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:58.021443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8547dd-17f3-46f2-b063-4f249bac6508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:58.072124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4b8547dd-17f3-46f2-b063-4f249bac6508 None None] MFA Rules not processed for user `2b2065bb4a1c4ba7b435d1ce53eca71e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:53:58.074010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4b8547dd-17f3-46f2-b063-4f249bac6508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:58.136194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:58.138896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:58.170391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1308/2617] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:53:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:53:58.188890 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8547dd-17f3-46f2-b063-4f249bac6508 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:58.191210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4b8547dd-17f3-46f2-b063-4f249bac6508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:58.199960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:58.246536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:58.270422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb0dd-ff99-452f-8c37-88aedf861561 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:58.277499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:58.278351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:58.278620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:58.278899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] PATH_INFO: `/v3/projects/f655224fff6c44b8a8277fcea0d1657d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:58.280546 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f655224fff6c44b8a8277fcea0d1657d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:58.282929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:58.352806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f655224fff6c44b8a8277fcea0d1657d', 'request_id': 'req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.358363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f655224fff6c44b8a8277fcea0d1657d', 'request_id': 'req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.363735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8547dd-17f3-46f2-b063-4f249bac6508 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:58.368320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f655224fff6c44b8a8277fcea0d1657d', 'request_id': 'req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.387026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] The token cache is being invalidate because project f655224fff6c44b8a8277fcea0d1657d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:53:58.387356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.388127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.390214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1310/2618] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:53:58 2022] DELETE /identity/v3/projects/f655224fff6c44b8a8277fcea0d1657d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:53:58.395961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:58.397535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:58.472857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:58.474400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:58.475169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:58.475947 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:58.479669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:53:58.511227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51811c7d-cc54-4c3d-9dc1-ee4e11e2b5c5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:58.517016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:58.518186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:58.518186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:58.518553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] PATH_INFO: `/v3/users/3207e35a6e924b9380a1ea4412ab5998` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:58.520376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3207e35a6e924b9380a1ea4412ab5998)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:58.522600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:58.644723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3207e35a6e924b9380a1ea4412ab5998'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.645948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:58.647788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3207e35a6e924b9380a1ea4412ab5998'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.650311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:58.661922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:58.669900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1309/2619] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:53:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:53:58.670845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3207e35a6e924b9380a1ea4412ab5998'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.684903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3207e35a6e924b9380a1ea4412ab5998'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.707212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4ce1b708-3e5d-4c90-badf-a83d3bc8501c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3207e35a6e924b9380a1ea4412ab5998'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:58.718348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1311/2620] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:53:58 2022] DELETE /identity/v3/users/3207e35a6e924b9380a1ea4412ab5998 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:53:59.380548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:53:59.397264 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da3614-4748-42fc-b6fe-ed78cd494e8b service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:53:59.693563 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:53:59.694918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:59.696646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:59.697486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] PATH_INFO: `/v3/projects/151a7b42e4874d3e862dbb278abf3171` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:53:59.700001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=151a7b42e4874d3e862dbb278abf3171)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:53:59.704402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:53:59.793535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:53:59.863838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151a7b42e4874d3e862dbb278abf3171', 'request_id': 'req-75f07015-8f22-4ca8-bf10-5cb1982501f0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:59.897961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151a7b42e4874d3e862dbb278abf3171', 'request_id': 'req-75f07015-8f22-4ca8-bf10-5cb1982501f0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:59.925894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151a7b42e4874d3e862dbb278abf3171', 'request_id': 'req-75f07015-8f22-4ca8-bf10-5cb1982501f0'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:59.979524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] The token cache is being invalidate because project 151a7b42e4874d3e862dbb278abf3171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:53:59.980298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:59.981644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75f07015-8f22-4ca8-bf10-5cb1982501f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:53:59.985598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1310/2621] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:53:59 2022] DELETE /identity/v3/projects/151a7b42e4874d3e862dbb278abf3171 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:53:59.996401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a030084-a7fb-4fd4-8eba-4927210bd809 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:53:59.997917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a030084-a7fb-4fd4-8eba-4927210bd809 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:53:59.998630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a030084-a7fb-4fd4-8eba-4927210bd809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.075345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1a030084-a7fb-4fd4-8eba-4927210bd809 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:00.077536 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1a030084-a7fb-4fd4-8eba-4927210bd809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:00.245577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1312/2622] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:00.254713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0587d4c-9b49-4216-8344-e8cd2fd5b1ad None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:00.255077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0587d4c-9b49-4216-8344-e8cd2fd5b1ad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:00.255470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0587d4c-9b49-4216-8344-e8cd2fd5b1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.281974 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b0587d4c-9b49-4216-8344-e8cd2fd5b1ad None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:00.283394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b0587d4c-9b49-4216-8344-e8cd2fd5b1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:00.298349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a030084-a7fb-4fd4-8eba-4927210bd809 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:00.299080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1311/2623] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:00.311738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a030084-a7fb-4fd4-8eba-4927210bd809 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:00.317627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:00.318362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:00.318771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:00.319131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.320071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:00.320448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:00.322197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:00.343148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0587d4c-9b49-4216-8344-e8cd2fd5b1ad None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:00.354643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0587d4c-9b49-4216-8344-e8cd2fd5b1ad None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:00.355734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1313/2624] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:00.360329 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d7785-b5f6-4e69-8b8f-55579efe30a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:00.360966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d7785-b5f6-4e69-8b8f-55579efe30a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:00.361239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d7785-b5f6-4e69-8b8f-55579efe30a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:00.361541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d7785-b5f6-4e69-8b8f-55579efe30a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.362201 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d7785-b5f6-4e69-8b8f-55579efe30a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:00.364391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d7785-b5f6-4e69-8b8f-55579efe30a6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:00.405059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:00.413439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1312/2625] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:00 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:00.420291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850a770-ed09-4c5e-b958-00008e721702 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:00.547630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63478c68-d4f5-466a-8e8e-fda2e567b0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:00.548399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63478c68-d4f5-466a-8e8e-fda2e567b0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:00.548858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63478c68-d4f5-466a-8e8e-fda2e567b0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:00.549433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63478c68-d4f5-466a-8e8e-fda2e567b0a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.550190 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63478c68-d4f5-466a-8e8e-fda2e567b0a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:00.552762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63478c68-d4f5-466a-8e8e-fda2e567b0a2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:00.592261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1314/2626] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:00 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:00.597245 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d7785-b5f6-4e69-8b8f-55579efe30a6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:00.618467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d7785-b5f6-4e69-8b8f-55579efe30a6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:00.632652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befeec4-8b05-4844-af8c-7b2bf47d2611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:00.636897 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befeec4-8b05-4844-af8c-7b2bf47d2611 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:00.640381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befeec4-8b05-4844-af8c-7b2bf47d2611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:00.642384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befeec4-8b05-4844-af8c-7b2bf47d2611 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.644053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befeec4-8b05-4844-af8c-7b2bf47d2611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:00.649362 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befeec4-8b05-4844-af8c-7b2bf47d2611 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:00.681795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1313/2627] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:00.684278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63478c68-d4f5-466a-8e8e-fda2e567b0a2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:00.709522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63478c68-d4f5-466a-8e8e-fda2e567b0a2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:00.719729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079307a-159b-4d32-97a1-9ac87a2cc097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:00.721073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079307a-159b-4d32-97a1-9ac87a2cc097 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:00.722308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079307a-159b-4d32-97a1-9ac87a2cc097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:00.762428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079307a-159b-4d32-97a1-9ac87a2cc097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.762428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079307a-159b-4d32-97a1-9ac87a2cc097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:00.762428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079307a-159b-4d32-97a1-9ac87a2cc097 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:00.764479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1315/2628] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:00.774269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befeec4-8b05-4844-af8c-7b2bf47d2611 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:00.796011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befeec4-8b05-4844-af8c-7b2bf47d2611 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:00.805885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fddaf-8185-495f-b149-13043558bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:00.808078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fddaf-8185-495f-b149-13043558bc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:00.808078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fddaf-8185-495f-b149-13043558bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:00.808475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fddaf-8185-495f-b149-13043558bc03 admin admin] PATH_INFO: `/v3/projects/f7f3a604cab342b392d4e079340bc1c5/users/74f2806844014eaaa91d2141985d29df/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.842417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fddaf-8185-495f-b149-13043558bc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7f3a604cab342b392d4e079340bc1c5, user_id=74f2806844014eaaa91d2141985d29df, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:00.849091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fddaf-8185-495f-b149-13043558bc03 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:00.903470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1314/2629] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:00 2022] PUT /identity/v3/projects/f7f3a604cab342b392d4e079340bc1c5/users/74f2806844014eaaa91d2141985d29df/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:00.917880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e4fbe-5a35-492f-b8e3-1cbd5500256b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:00.918527 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e4fbe-5a35-492f-b8e3-1cbd5500256b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:00.919205 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e4fbe-5a35-492f-b8e3-1cbd5500256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:00.975187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-705e4fbe-5a35-492f-b8e3-1cbd5500256b None None] MFA Rules not processed for user `74f2806844014eaaa91d2141985d29df`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:00.977605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-705e4fbe-5a35-492f-b8e3-1cbd5500256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:01.044135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:01.118829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1316/2630] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:01.123681 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fddaf-8185-495f-b149-13043558bc03 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:01.284650 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fddaf-8185-495f-b149-13043558bc03 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:01.293921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15c36-147e-4368-8f36-40a0fe858c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:01.293921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15c36-147e-4368-8f36-40a0fe858c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:01.294483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15c36-147e-4368-8f36-40a0fe858c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:01.294636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15c36-147e-4368-8f36-40a0fe858c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:01.296041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b15c36-147e-4368-8f36-40a0fe858c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:01.298211 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b15c36-147e-4368-8f36-40a0fe858c01 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:01.354660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1315/2631] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:01 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:01.359073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e4fbe-5a35-492f-b8e3-1cbd5500256b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:01.384998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e4fbe-5a35-492f-b8e3-1cbd5500256b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:01.638771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee9e3a-a7f2-4672-941b-186c2150d4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:01.642005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee9e3a-a7f2-4672-941b-186c2150d4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:01.642978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee9e3a-a7f2-4672-941b-186c2150d4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:01.643839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee9e3a-a7f2-4672-941b-186c2150d4d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:01.646389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee9e3a-a7f2-4672-941b-186c2150d4d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:01.651763 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee9e3a-a7f2-4672-941b-186c2150d4d1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:01.743183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1317/2632] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:01 2022] POST /identity/v3/users => generated 300 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:01.746005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15c36-147e-4368-8f36-40a0fe858c01 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:01.781063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15c36-147e-4368-8f36-40a0fe858c01 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:01.801552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e904cea-61cd-4112-9518-5595a1d651d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:01.819155 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904cea-61cd-4112-9518-5595a1d651d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:01.820672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904cea-61cd-4112-9518-5595a1d651d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:01.823392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904cea-61cd-4112-9518-5595a1d651d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:01.826874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e904cea-61cd-4112-9518-5595a1d651d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:01.835252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e904cea-61cd-4112-9518-5595a1d651d1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:01.875718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1316/2633] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:01.883756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee9e3a-a7f2-4672-941b-186c2150d4d1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:01.927091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee9e3a-a7f2-4672-941b-186c2150d4d1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:01.940060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59130bb-4c66-4b75-a70c-fa5789fc7433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:01.941729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59130bb-4c66-4b75-a70c-fa5789fc7433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:01.942357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59130bb-4c66-4b75-a70c-fa5789fc7433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:01.943058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59130bb-4c66-4b75-a70c-fa5789fc7433 admin admin] PATH_INFO: `/v3/projects/bf84f2d896e742cca6e1aa08f61d275b/users/cb86ad5573e34e59b39e7547ebba619b/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:02.004957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59130bb-4c66-4b75-a70c-fa5789fc7433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf84f2d896e742cca6e1aa08f61d275b, user_id=cb86ad5573e34e59b39e7547ebba619b, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:02.013377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59130bb-4c66-4b75-a70c-fa5789fc7433 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:02.048525 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1318/2634] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:01 2022] PUT /identity/v3/projects/bf84f2d896e742cca6e1aa08f61d275b/users/cb86ad5573e34e59b39e7547ebba619b/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:02.054783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e904cea-61cd-4112-9518-5595a1d651d1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:02.069396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e904cea-61cd-4112-9518-5595a1d651d1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:02.095493 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:02.176289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40660b9c-ce37-435e-8965-4316c0620ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:02.176922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40660b9c-ce37-435e-8965-4316c0620ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:02.177394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40660b9c-ce37-435e-8965-4316c0620ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:02.177840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40660b9c-ce37-435e-8965-4316c0620ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:02.178724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40660b9c-ce37-435e-8965-4316c0620ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:02.180455 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40660b9c-ce37-435e-8965-4316c0620ba7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:02.193553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1317/2635] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:54:02.198739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59130bb-4c66-4b75-a70c-fa5789fc7433 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:02.214170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59130bb-4c66-4b75-a70c-fa5789fc7433 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:02.219148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:02.220029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:02.220310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:02.220578 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] PATH_INFO: `/v3/projects/bf84f2d896e742cca6e1aa08f61d275b/users/cb86ad5573e34e59b39e7547ebba619b/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:02.223079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf84f2d896e742cca6e1aa08f61d275b, user_id=cb86ad5573e34e59b39e7547ebba619b, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:02.225864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:02.238018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1319/2636] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:02 2022] PUT /identity/v3/projects/bf84f2d896e742cca6e1aa08f61d275b/users/cb86ad5573e34e59b39e7547ebba619b/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:02.250187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9cb23-bd89-4377-98a0-57fa722a846e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:02.250579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9cb23-bd89-4377-98a0-57fa722a846e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:02.251055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9cb23-bd89-4377-98a0-57fa722a846e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:02.298043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-72a9cb23-bd89-4377-98a0-57fa722a846e None None] MFA Rules not processed for user `cb86ad5573e34e59b39e7547ebba619b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:02.300527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-72a9cb23-bd89-4377-98a0-57fa722a846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:02.426551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1318/2637] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:02.452040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:02.455558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:02.610105 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8762cf-dd67-4e2e-ae45-b300e106084c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:02.615013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d87b-1e76-41f3-aaa2-fea1d65ddaf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:02.615819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d87b-1e76-41f3-aaa2-fea1d65ddaf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:02.616219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d87b-1e76-41f3-aaa2-fea1d65ddaf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:02.616812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d87b-1e76-41f3-aaa2-fea1d65ddaf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:02.627932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50d87b-1e76-41f3-aaa2-fea1d65ddaf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:02.629876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50d87b-1e76-41f3-aaa2-fea1d65ddaf8 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:02.642846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1320/2638] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:03.457698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9cb23-bd89-4377-98a0-57fa722a846e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:03.460127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-72a9cb23-bd89-4377-98a0-57fa722a846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:03.548552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9cb23-bd89-4377-98a0-57fa722a846e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:03.557051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:03.558050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:03.558512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:03.558999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] PATH_INFO: `/v3/users/4c3e5c1776b8470a9b677bac8dfbc9e6` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:03.561222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3e5c1776b8470a9b677bac8dfbc9e6)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:03.565193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:03.601264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d87b-1e76-41f3-aaa2-fea1d65ddaf8 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:03.613912 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d87b-1e76-41f3-aaa2-fea1d65ddaf8 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:03.623581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:03.624873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:03.625320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:03.625776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] PATH_INFO: `/v3/users/74f2806844014eaaa91d2141985d29df` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:03.627566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74f2806844014eaaa91d2141985d29df)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:03.630300 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:03.648583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:03.659740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3e5c1776b8470a9b677bac8dfbc9e6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.660814 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3e5c1776b8470a9b677bac8dfbc9e6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.666887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:03.672289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3e5c1776b8470a9b677bac8dfbc9e6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.675419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3e5c1776b8470a9b677bac8dfbc9e6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.696247 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c9766efc-934f-4298-b320-787f3c61f7a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3e5c1776b8470a9b677bac8dfbc9e6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.708487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74f2806844014eaaa91d2141985d29df'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.709501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74f2806844014eaaa91d2141985d29df'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.718471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1319/2639] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:03 2022] DELETE /identity/v3/users/4c3e5c1776b8470a9b677bac8dfbc9e6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:03.731440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74f2806844014eaaa91d2141985d29df'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.746284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74f2806844014eaaa91d2141985d29df'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.758264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74f2806844014eaaa91d2141985d29df'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:03.766264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1321/2640] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:03 2022] DELETE /identity/v3/users/74f2806844014eaaa91d2141985d29df => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:03.771262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977d922-1af1-4d70-bed6-8321349b50b9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:03.772011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977d922-1af1-4d70-bed6-8321349b50b9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:03.772011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977d922-1af1-4d70-bed6-8321349b50b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:03.796232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b977d922-1af1-4d70-bed6-8321349b50b9 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:03.797584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b977d922-1af1-4d70-bed6-8321349b50b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:03.903195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1320/2641] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:03.916284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:03.918548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:03.988153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977d922-1af1-4d70-bed6-8321349b50b9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:03.990920 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b977d922-1af1-4d70-bed6-8321349b50b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:04.009009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977d922-1af1-4d70-bed6-8321349b50b9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.015287 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.015500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.015657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.015983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] PATH_INFO: `/v3/projects/16f3955065954f8f860bada6261223c4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.018217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f3955065954f8f860bada6261223c4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.019476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.021230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd39d88-f15f-43a9-8ce6-7fe3bc888898 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.025569 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b931c8-aca3-45f4-b248-b146bc5fb597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.026366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b931c8-aca3-45f4-b248-b146bc5fb597 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.026638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b931c8-aca3-45f4-b248-b146bc5fb597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.026976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b931c8-aca3-45f4-b248-b146bc5fb597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.039597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b931c8-aca3-45f4-b248-b146bc5fb597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.041255 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b931c8-aca3-45f4-b248-b146bc5fb597 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.056022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1322/2642] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:04.092980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f3955065954f8f860bada6261223c4', 'request_id': 'req-bb0f0609-1d2f-47ff-b04f-362c12849b97'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.106085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f3955065954f8f860bada6261223c4', 'request_id': 'req-bb0f0609-1d2f-47ff-b04f-362c12849b97'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.118130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f3955065954f8f860bada6261223c4', 'request_id': 'req-bb0f0609-1d2f-47ff-b04f-362c12849b97'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.133344 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] The token cache is being invalidate because project 16f3955065954f8f860bada6261223c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:04.133799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.134794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bb0f0609-1d2f-47ff-b04f-362c12849b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.137043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1321/2643] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:03 2022] DELETE /identity/v3/projects/16f3955065954f8f860bada6261223c4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:04.145865 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c73fa9-6465-4227-b544-2710dcc53e9b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.146227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c73fa9-6465-4227-b544-2710dcc53e9b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.146863 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c73fa9-6465-4227-b544-2710dcc53e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.174060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f3c73fa9-6465-4227-b544-2710dcc53e9b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:04.175596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f3c73fa9-6465-4227-b544-2710dcc53e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:04.279642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1323/2644] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:04.287589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfbe73-4f20-491d-bb6b-685aab9d92d7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.287915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfbe73-4f20-491d-bb6b-685aab9d92d7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.288292 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfbe73-4f20-491d-bb6b-685aab9d92d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.313207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6edfbe73-4f20-491d-bb6b-685aab9d92d7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:04.314574 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6edfbe73-4f20-491d-bb6b-685aab9d92d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:04.324426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1322/2645] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:04.329321 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c73fa9-6465-4227-b544-2710dcc53e9b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:04.342295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c73fa9-6465-4227-b544-2710dcc53e9b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.349118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.349885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.350256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.350635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.351588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:04.351998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.353760 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.373558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1324/2646] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:04.377660 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfbe73-4f20-491d-bb6b-685aab9d92d7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:04.395282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfbe73-4f20-491d-bb6b-685aab9d92d7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.400426 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.401241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.401583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.402016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.402831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.404962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.438504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1323/2647] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:04 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:04.445961 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:04.454688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:04.457085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:04.466600 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb0e80-75d5-48a0-8455-c4e54a2b6b89 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.474644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:04.517054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:04.556107 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bbd69-7b80-4c93-a89b-2301eb60d4b7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.560753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f0c80-f620-45fa-a105-b54d9f66c1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.561404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f0c80-f620-45fa-a105-b54d9f66c1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.561593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.561753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f0c80-f620-45fa-a105-b54d9f66c1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.562052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.562203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f0c80-f620-45fa-a105-b54d9f66c1d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.562335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.562582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] PATH_INFO: `/v3/projects/f7f3a604cab342b392d4e079340bc1c5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.562954 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f0c80-f620-45fa-a105-b54d9f66c1d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.563896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7f3a604cab342b392d4e079340bc1c5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.565347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f0c80-f620-45fa-a105-b54d9f66c1d9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.566078 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.595141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1325/2648] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:04 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:04.601319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f0c80-f620-45fa-a105-b54d9f66c1d9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:04.616753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f0c80-f620-45fa-a105-b54d9f66c1d9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.628453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7f3a604cab342b392d4e079340bc1c5', 'request_id': 'req-a3d002e8-4655-4a33-b02c-1a8492a379e5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.638324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7f3a604cab342b392d4e079340bc1c5', 'request_id': 'req-a3d002e8-4655-4a33-b02c-1a8492a379e5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.648960 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7f3a604cab342b392d4e079340bc1c5', 'request_id': 'req-a3d002e8-4655-4a33-b02c-1a8492a379e5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.661718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] The token cache is being invalidate because project f7f3a604cab342b392d4e079340bc1c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:04.662009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.662698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.664330 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1324/2649] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:04 2022] DELETE /identity/v3/projects/f7f3a604cab342b392d4e079340bc1c5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:04.668788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:04.670415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:04.700605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.701235 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.701510 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.701782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.702451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.704154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.718991 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1326/2650] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:04.720296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:04.722685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:04.742984 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b26c3-c110-49bb-b601-22a327570c4a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.749472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6b95-f7a4-4b80-9612-7e7c29ff9654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.750294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6b95-f7a4-4b80-9612-7e7c29ff9654 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.750739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6b95-f7a4-4b80-9612-7e7c29ff9654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.751013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6b95-f7a4-4b80-9612-7e7c29ff9654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.751735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce6b95-f7a4-4b80-9612-7e7c29ff9654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.753941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce6b95-f7a4-4b80-9612-7e7c29ff9654 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.760522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d002e8-4655-4a33-b02c-1a8492a379e5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.766528 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.767339 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.767655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.768045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] PATH_INFO: `/v3/users/cb86ad5573e34e59b39e7547ebba619b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.769382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb86ad5573e34e59b39e7547ebba619b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.771612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.775288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1327/2651] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:54:04.780852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6b95-f7a4-4b80-9612-7e7c29ff9654 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:04.798820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6b95-f7a4-4b80-9612-7e7c29ff9654 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:04.806955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4106e-974f-42b0-8d2b-1f0500ad5285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:04.808141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4106e-974f-42b0-8d2b-1f0500ad5285 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.808544 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4106e-974f-42b0-8d2b-1f0500ad5285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.808958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4106e-974f-42b0-8d2b-1f0500ad5285 admin admin] PATH_INFO: `/v3/projects/63bc257821784396863250b764e6af8e/users/c0feb5a97c6a4e33861173c19b0d469f/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.831020 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4106e-974f-42b0-8d2b-1f0500ad5285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63bc257821784396863250b764e6af8e, user_id=c0feb5a97c6a4e33861173c19b0d469f, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:04.832718 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:04.833899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4106e-974f-42b0-8d2b-1f0500ad5285 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:04.834917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb86ad5573e34e59b39e7547ebba619b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.835806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb86ad5573e34e59b39e7547ebba619b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.847316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb86ad5573e34e59b39e7547ebba619b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.848538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb86ad5573e34e59b39e7547ebba619b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.852431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1328/2652] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:04 2022] PUT /identity/v3/projects/63bc257821784396863250b764e6af8e/users/c0feb5a97c6a4e33861173c19b0d469f/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:04.863429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d718-455b-43ab-bb13-010826da2649 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:04.864008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d718-455b-43ab-bb13-010826da2649 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:04.864299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d718-455b-43ab-bb13-010826da2649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:04.872170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb86ad5573e34e59b39e7547ebba619b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:04.902882 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1325/2653] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:04 2022] DELETE /identity/v3/users/cb86ad5573e34e59b39e7547ebba619b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:04.921195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8847d718-455b-43ab-bb13-010826da2649 None None] MFA Rules not processed for user `c0feb5a97c6a4e33861173c19b0d469f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:04.922438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8847d718-455b-43ab-bb13-010826da2649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:05.022664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1329/2654] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:05.037497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:05.039049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:05.135488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b0b59-0fbe-4ef4-8d81-4c1d040baa76 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:05.140653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:05.141293 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:05.141575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:05.142049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:05.164104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:05.166786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:05.188962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1326/2655] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:05.319774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847d718-455b-43ab-bb13-010826da2649 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:05.321626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8847d718-455b-43ab-bb13-010826da2649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:05.335823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:05.439963 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847d718-455b-43ab-bb13-010826da2649 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:05.456223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:05.457316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:05.457810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:05.458392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] PATH_INFO: `/v3/projects/bf84f2d896e742cca6e1aa08f61d275b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:05.459893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf84f2d896e742cca6e1aa08f61d275b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:05.461941 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:05.530913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf84f2d896e742cca6e1aa08f61d275b', 'request_id': 'req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:05.538472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf84f2d896e742cca6e1aa08f61d275b', 'request_id': 'req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:05.547337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf84f2d896e742cca6e1aa08f61d275b', 'request_id': 'req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:05.566400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:05.568881 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:05.583741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] The token cache is being invalidate because project bf84f2d896e742cca6e1aa08f61d275b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:05.584283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:05.585617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f50f471a-0f9a-4696-a38b-3ffd44a45fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:05.595519 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1330/2656] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:05 2022] DELETE /identity/v3/projects/bf84f2d896e742cca6e1aa08f61d275b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:05.597768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84193bd-df23-4e75-9f13-e985c9c901fd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:05.598222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84193bd-df23-4e75-9f13-e985c9c901fd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:05.598665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84193bd-df23-4e75-9f13-e985c9c901fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:05.638497 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a84193bd-df23-4e75-9f13-e985c9c901fd None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:05.640061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a84193bd-df23-4e75-9f13-e985c9c901fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:05.712561 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b8385-8992-47b0-b2bf-c88c7dee1b9f service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:05.727279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1331/2657] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:05.738646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25564d1-320c-4143-9b85-4054c98ecf76 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:05.739322 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25564d1-320c-4143-9b85-4054c98ecf76 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:05.739812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25564d1-320c-4143-9b85-4054c98ecf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:05.770630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f25564d1-320c-4143-9b85-4054c98ecf76 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:05.772030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f25564d1-320c-4143-9b85-4054c98ecf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:05.787752 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1332/2658] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:05.792618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25564d1-320c-4143-9b85-4054c98ecf76 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:05.810216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25564d1-320c-4143-9b85-4054c98ecf76 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:05.817016 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:05.817684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:05.817966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:05.818293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:05.818959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:05.819295 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:05.821004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:05.829658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:05.830783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:05.831301 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:05.831786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:05.835268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:05.835446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1333/2659] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:05.839885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:05.851075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27242d1-dbab-4e3b-b93f-92419ead41f3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:05.856215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f962dd-a200-472a-b8f0-b4fe0fbc5a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:05.856839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f962dd-a200-472a-b8f0-b4fe0fbc5a31 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:05.857118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f962dd-a200-472a-b8f0-b4fe0fbc5a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:05.857385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f962dd-a200-472a-b8f0-b4fe0fbc5a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:05.857489 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:05.858055 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f962dd-a200-472a-b8f0-b4fe0fbc5a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:05.860071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f962dd-a200-472a-b8f0-b4fe0fbc5a31 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:05.861213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:05.884458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1327/2660] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:05.897049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:05.913351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1334/2661] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:05 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:05.917559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:05.929773 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b6ab6-430a-420f-86a8-490279c18f5f service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:05.986271 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:06.064909 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5135c1-c908-4db2-b57a-cd33615f11bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:06.065616 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5135c1-c908-4db2-b57a-cd33615f11bc admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.066025 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5135c1-c908-4db2-b57a-cd33615f11bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.066396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5135c1-c908-4db2-b57a-cd33615f11bc admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.067145 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5135c1-c908-4db2-b57a-cd33615f11bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:06.069595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5135c1-c908-4db2-b57a-cd33615f11bc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:06.103706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1328/2662] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:05 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:06.104792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f962dd-a200-472a-b8f0-b4fe0fbc5a31 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:06.119459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f962dd-a200-472a-b8f0-b4fe0fbc5a31 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:06.125208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f03201-b248-499e-b72c-1c6d46c4555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:06.126433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f03201-b248-499e-b72c-1c6d46c4555a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.126988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f03201-b248-499e-b72c-1c6d46c4555a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.127643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f03201-b248-499e-b72c-1c6d46c4555a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.128603 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f03201-b248-499e-b72c-1c6d46c4555a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:06.130555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f03201-b248-499e-b72c-1c6d46c4555a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:06.147212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1335/2663] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:06.152064 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5135c1-c908-4db2-b57a-cd33615f11bc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:06.171530 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5135c1-c908-4db2-b57a-cd33615f11bc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:06.184734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49837c-25a8-488b-8943-bc39acfbc0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:06.185747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49837c-25a8-488b-8943-bc39acfbc0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.186285 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49837c-25a8-488b-8943-bc39acfbc0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.186793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49837c-25a8-488b-8943-bc39acfbc0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.188015 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49837c-25a8-488b-8943-bc39acfbc0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:06.191016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49837c-25a8-488b-8943-bc39acfbc0c5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:06.209925 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1329/2664] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:54:06.215098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f03201-b248-499e-b72c-1c6d46c4555a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:06.232638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f03201-b248-499e-b72c-1c6d46c4555a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:06.238172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dfc41-d49f-4def-b17f-21ad476b0936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:06.238857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dfc41-d49f-4def-b17f-21ad476b0936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.239137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dfc41-d49f-4def-b17f-21ad476b0936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.239447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dfc41-d49f-4def-b17f-21ad476b0936 admin admin] PATH_INFO: `/v3/projects/03885b28674e445ebad72405a334f1f7/users/bd03ff340761414885f054ccb2cd9eab/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.256565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dfc41-d49f-4def-b17f-21ad476b0936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03885b28674e445ebad72405a334f1f7, user_id=bd03ff340761414885f054ccb2cd9eab, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:06.259365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dfc41-d49f-4def-b17f-21ad476b0936 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:06.276758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1336/2665] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:06 2022] PUT /identity/v3/projects/03885b28674e445ebad72405a334f1f7/users/bd03ff340761414885f054ccb2cd9eab/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:06.288129 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bdbcd-e9f6-465a-80eb-8cce6a76637c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.288794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bdbcd-e9f6-465a-80eb-8cce6a76637c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.289322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bdbcd-e9f6-465a-80eb-8cce6a76637c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.324841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1d9bdbcd-e9f6-465a-80eb-8cce6a76637c None None] MFA Rules not processed for user `bd03ff340761414885f054ccb2cd9eab`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:06.326091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1d9bdbcd-e9f6-465a-80eb-8cce6a76637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:06.432222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1330/2666] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:06.438115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dfc41-d49f-4def-b17f-21ad476b0936 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:06.556010 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dfc41-d49f-4def-b17f-21ad476b0936 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:06.561557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371312-2c30-4ddf-941f-ea1105b9bf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:06.562302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371312-2c30-4ddf-941f-ea1105b9bf8d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.562724 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371312-2c30-4ddf-941f-ea1105b9bf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.563073 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371312-2c30-4ddf-941f-ea1105b9bf8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.563849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69371312-2c30-4ddf-941f-ea1105b9bf8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:06.566040 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69371312-2c30-4ddf-941f-ea1105b9bf8d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:06.606478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1337/2667] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:06 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:06.608096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bdbcd-e9f6-465a-80eb-8cce6a76637c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:06.620861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bdbcd-e9f6-465a-80eb-8cce6a76637c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:06.711675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fe6ad-b769-489e-8f98-53f78f42cbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:06.712183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fe6ad-b769-489e-8f98-53f78f42cbbe admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.712459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fe6ad-b769-489e-8f98-53f78f42cbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.712738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fe6ad-b769-489e-8f98-53f78f42cbbe admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.713379 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fe6ad-b769-489e-8f98-53f78f42cbbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:06.715492 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fe6ad-b769-489e-8f98-53f78f42cbbe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:06.742556 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1331/2668] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:06 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:06.748056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371312-2c30-4ddf-941f-ea1105b9bf8d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:06.761135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371312-2c30-4ddf-941f-ea1105b9bf8d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:06.768153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cad281-d22c-45e7-a9bc-4fb1818647de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:06.768798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cad281-d22c-45e7-a9bc-4fb1818647de admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.769082 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cad281-d22c-45e7-a9bc-4fb1818647de admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.769366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cad281-d22c-45e7-a9bc-4fb1818647de admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.770166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cad281-d22c-45e7-a9bc-4fb1818647de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:06.772014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cad281-d22c-45e7-a9bc-4fb1818647de admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:06.787301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1338/2669] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:06.790519 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fe6ad-b769-489e-8f98-53f78f42cbbe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:06.807126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fe6ad-b769-489e-8f98-53f78f42cbbe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:06.814497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d9e7d-57bb-4b55-a7fd-9b3dc370899b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:06.815597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d9e7d-57bb-4b55-a7fd-9b3dc370899b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:06.816004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d9e7d-57bb-4b55-a7fd-9b3dc370899b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:06.816403 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d9e7d-57bb-4b55-a7fd-9b3dc370899b admin admin] PATH_INFO: `/v3/projects/2a243587231b46a1a92960f94100f32c/users/3d7d9eb386c74be89f083cfa3e9c1f4c/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:06.842859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d9e7d-57bb-4b55-a7fd-9b3dc370899b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a243587231b46a1a92960f94100f32c, user_id=3d7d9eb386c74be89f083cfa3e9c1f4c, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:06.849087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d9e7d-57bb-4b55-a7fd-9b3dc370899b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:06.865032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:06.879552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1332/2670] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:06 2022] PUT /identity/v3/projects/2a243587231b46a1a92960f94100f32c/users/3d7d9eb386c74be89f083cfa3e9c1f4c/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:06.886046 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cad281-d22c-45e7-a9bc-4fb1818647de admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:06.904700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cad281-d22c-45e7-a9bc-4fb1818647de admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:07.013710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2fe46-60dd-479c-8d7b-9e00c2d106db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:07.014466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2fe46-60dd-479c-8d7b-9e00c2d106db admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:07.014827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2fe46-60dd-479c-8d7b-9e00c2d106db admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:07.015164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2fe46-60dd-479c-8d7b-9e00c2d106db admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:07.015949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2fe46-60dd-479c-8d7b-9e00c2d106db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:07.017771 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2fe46-60dd-479c-8d7b-9e00c2d106db admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:07.031270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1339/2671] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:07.037296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d9e7d-57bb-4b55-a7fd-9b3dc370899b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:07.052309 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d9e7d-57bb-4b55-a7fd-9b3dc370899b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:07.058434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:07.059356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:07.059656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:07.059993 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] PATH_INFO: `/v3/projects/2a243587231b46a1a92960f94100f32c/users/3d7d9eb386c74be89f083cfa3e9c1f4c/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:07.062999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a243587231b46a1a92960f94100f32c, user_id=3d7d9eb386c74be89f083cfa3e9c1f4c, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:07.067443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:07.087656 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1333/2672] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:07 2022] PUT /identity/v3/projects/2a243587231b46a1a92960f94100f32c/users/3d7d9eb386c74be89f083cfa3e9c1f4c/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:07.105417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf2f8-95d6-44d5-b8b6-2c6c5707bda3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:07.107142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf2f8-95d6-44d5-b8b6-2c6c5707bda3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:07.107451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf2f8-95d6-44d5-b8b6-2c6c5707bda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:07.149882 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-69ddf2f8-95d6-44d5-b8b6-2c6c5707bda3 None None] MFA Rules not processed for user `3d7d9eb386c74be89f083cfa3e9c1f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:07.152201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69ddf2f8-95d6-44d5-b8b6-2c6c5707bda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:07.202003 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:07.326469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1340/2673] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:07.355314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:07.358522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:07.525599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593eaa5-d5b5-466f-85e5-f07db90513de admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:07.532011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259caefb-b9b5-4032-b686-01ade667badf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:07.532742 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259caefb-b9b5-4032-b686-01ade667badf service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:07.533083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259caefb-b9b5-4032-b686-01ade667badf service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:07.533631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259caefb-b9b5-4032-b686-01ade667badf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:07.631985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259caefb-b9b5-4032-b686-01ade667badf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:07.635992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259caefb-b9b5-4032-b686-01ade667badf service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:07.655719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1334/2674] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:08.692023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddf2f8-95d6-44d5-b8b6-2c6c5707bda3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:08.693511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69ddf2f8-95d6-44d5-b8b6-2c6c5707bda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:08.813669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddf2f8-95d6-44d5-b8b6-2c6c5707bda3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:08.823988 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:08.826084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:08.826837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:08.827700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] PATH_INFO: `/v3/users/2b2065bb4a1c4ba7b435d1ce53eca71e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:08.830447 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b2065bb4a1c4ba7b435d1ce53eca71e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:08.835039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:08.964969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b2065bb4a1c4ba7b435d1ce53eca71e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:08.966246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b2065bb4a1c4ba7b435d1ce53eca71e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:08.997491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b2065bb4a1c4ba7b435d1ce53eca71e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:09.017537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b2065bb4a1c4ba7b435d1ce53eca71e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:09.038011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b2065bb4a1c4ba7b435d1ce53eca71e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:09.058920 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1341/2675] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:08 2022] DELETE /identity/v3/users/2b2065bb4a1c4ba7b435d1ce53eca71e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:09.061893 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0078-3a6c-4083-bc38-f2e98cc7de80 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:09.062463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0078-3a6c-4083-bc38-f2e98cc7de80 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:09.062463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0078-3a6c-4083-bc38-f2e98cc7de80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:09.095517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fada0078-3a6c-4083-bc38-f2e98cc7de80 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:09.098412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fada0078-3a6c-4083-bc38-f2e98cc7de80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:09.257760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1335/2676] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:09.307481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:09.311294 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:09.456037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:09.503134 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8159271-2645-4a65-a6f1-282cb1ec50b9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:09.529901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:09.535928 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:09.537485 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:09.538238 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:09.559914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:09.564511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:09.587286 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1342/2677] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:10.197555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada0078-3a6c-4083-bc38-f2e98cc7de80 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:10.199154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fada0078-3a6c-4083-bc38-f2e98cc7de80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:10.220458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada0078-3a6c-4083-bc38-f2e98cc7de80 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:10.229034 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:10.229711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:10.230423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:10.233996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] PATH_INFO: `/v3/projects/35da2362b7d243feb77b2dda2f490b0c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:10.235704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35da2362b7d243feb77b2dda2f490b0c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:10.238429 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:10.319111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:10.334532 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35da2362b7d243feb77b2dda2f490b0c', 'request_id': 'req-fde52ea9-260d-4502-901a-d67e52c9e790'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:10.351147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35da2362b7d243feb77b2dda2f490b0c', 'request_id': 'req-fde52ea9-260d-4502-901a-d67e52c9e790'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:10.388562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35da2362b7d243feb77b2dda2f490b0c', 'request_id': 'req-fde52ea9-260d-4502-901a-d67e52c9e790'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:10.418860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] The token cache is being invalidate because project 35da2362b7d243feb77b2dda2f490b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:10.419630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:10.420593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:10.422475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1336/2678] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:10 2022] DELETE /identity/v3/projects/35da2362b7d243feb77b2dda2f490b0c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:11.239867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:11.244811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:11.291347 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd843a-5443-4077-9152-1f94db708d64 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:11.396776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:11.398388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:11.398857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:11.399411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] PATH_INFO: `/v3/users/a8320be5e9644b6699ac526913e109eb` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:11.418989 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8320be5e9644b6699ac526913e109eb)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:11.421026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:11.490214 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8320be5e9644b6699ac526913e109eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:11.491328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8320be5e9644b6699ac526913e109eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:11.508918 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8320be5e9644b6699ac526913e109eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:11.523923 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8320be5e9644b6699ac526913e109eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:11.535840 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8af76f93-ff7c-43d3-8a4c-1df2d859bb23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8320be5e9644b6699ac526913e109eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:11.548759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1343/2679] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:11 2022] DELETE /identity/v3/users/a8320be5e9644b6699ac526913e109eb => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:11.814184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:11.817539 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:11.875462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52ea9-260d-4502-901a-d67e52c9e790 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:12.039501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:12.041188 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.041668 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.042143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] PATH_INFO: `/v3/projects/8687bcf9d2a242b5af329bce4a8ad6d9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.044623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8687bcf9d2a242b5af329bce4a8ad6d9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:12.048529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:12.125467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8687bcf9d2a242b5af329bce4a8ad6d9', 'request_id': 'req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:12.134805 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8687bcf9d2a242b5af329bce4a8ad6d9', 'request_id': 'req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:12.146484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8687bcf9d2a242b5af329bce4a8ad6d9', 'request_id': 'req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:12.169724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] The token cache is being invalidate because project 8687bcf9d2a242b5af329bce4a8ad6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:12.170444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:12.171675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e690105a-226f-43a0-b5ca-f9c28cd5a8ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:12.174845 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1337/2680] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:11 2022] DELETE /identity/v3/projects/8687bcf9d2a242b5af329bce4a8ad6d9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:12.182924 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d513-66bb-4c8b-95c5-a13dce655bdc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.183422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d513-66bb-4c8b-95c5-a13dce655bdc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.183837 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d513-66bb-4c8b-95c5-a13dce655bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.211460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8604d513-66bb-4c8b-95c5-a13dce655bdc None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:12.212720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8604d513-66bb-4c8b-95c5-a13dce655bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:12.303191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:12.326766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1344/2681] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:12.340811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92602035-abfd-44a2-9d93-bb654edc75dd None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.341156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92602035-abfd-44a2-9d93-bb654edc75dd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.341488 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92602035-abfd-44a2-9d93-bb654edc75dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.365090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-92602035-abfd-44a2-9d93-bb654edc75dd None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:12.366358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-92602035-abfd-44a2-9d93-bb654edc75dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:12.378583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1338/2682] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:12.384573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604d513-66bb-4c8b-95c5-a13dce655bdc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:12.397357 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604d513-66bb-4c8b-95c5-a13dce655bdc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:12.403526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:12.404166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.404444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.404798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.405487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:12.405795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:12.408511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:12.424188 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1345/2683] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:12.430560 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92602035-abfd-44a2-9d93-bb654edc75dd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:12.443887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92602035-abfd-44a2-9d93-bb654edc75dd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:12.450477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8aab-ece3-4f21-a8a8-53aa9cac5d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:12.451158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8aab-ece3-4f21-a8a8-53aa9cac5d2b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.451613 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8aab-ece3-4f21-a8a8-53aa9cac5d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.451995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8aab-ece3-4f21-a8a8-53aa9cac5d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.452831 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc8aab-ece3-4f21-a8a8-53aa9cac5d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:12.455571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc8aab-ece3-4f21-a8a8-53aa9cac5d2b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:12.493719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1339/2684] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:12 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:12.499451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:12.514287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd0a8e-02d3-4cbb-9f03-bb1de7038c51 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:12.593208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e2b19-dc93-46ee-8cea-023a21a2f2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:12.593845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2b19-dc93-46ee-8cea-023a21a2f2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.594132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2b19-dc93-46ee-8cea-023a21a2f2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.594412 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2b19-dc93-46ee-8cea-023a21a2f2b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.595122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e2b19-dc93-46ee-8cea-023a21a2f2b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:12.598274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e2b19-dc93-46ee-8cea-023a21a2f2b8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:12.627650 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1346/2685] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:12 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:12.632849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8aab-ece3-4f21-a8a8-53aa9cac5d2b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:12.649759 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8aab-ece3-4f21-a8a8-53aa9cac5d2b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:12.655572 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eda16-ff14-45fd-8a89-699168390b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:12.656242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eda16-ff14-45fd-8a89-699168390b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.656621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eda16-ff14-45fd-8a89-699168390b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.656936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eda16-ff14-45fd-8a89-699168390b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.657719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eda16-ff14-45fd-8a89-699168390b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:12.660000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eda16-ff14-45fd-8a89-699168390b03 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:12.674152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1340/2686] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:12.679839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e2b19-dc93-46ee-8cea-023a21a2f2b8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:12.697424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e2b19-dc93-46ee-8cea-023a21a2f2b8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:12.704872 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e203221-3929-42b3-bff4-b97cc367ec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:12.705915 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e203221-3929-42b3-bff4-b97cc367ec67 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.706411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e203221-3929-42b3-bff4-b97cc367ec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.706906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e203221-3929-42b3-bff4-b97cc367ec67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.708148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e203221-3929-42b3-bff4-b97cc367ec67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:12.711655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e203221-3929-42b3-bff4-b97cc367ec67 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:12.733742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1347/2687] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:12.738545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eda16-ff14-45fd-8a89-699168390b03 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:12.750123 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eda16-ff14-45fd-8a89-699168390b03 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:12.755838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:12.756533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.756812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.757084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] PATH_INFO: `/v3/projects/23ff9f71e16d460781dd5889d784b4a3/users/454a193b26684746bc8c9278a1f1f175/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.770690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:12.776811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ff9f71e16d460781dd5889d784b4a3, user_id=454a193b26684746bc8c9278a1f1f175, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:12.780164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:12.797100 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1341/2688] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:12 2022] PUT /identity/v3/projects/23ff9f71e16d460781dd5889d784b4a3/users/454a193b26684746bc8c9278a1f1f175/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:12.803327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce215b28-2651-4735-8fb9-ea1d346a42e7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:12.803786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce215b28-2651-4735-8fb9-ea1d346a42e7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:12.804123 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce215b28-2651-4735-8fb9-ea1d346a42e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:12.826517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ce215b28-2651-4735-8fb9-ea1d346a42e7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:12.827803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ce215b28-2651-4735-8fb9-ea1d346a42e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:12.903094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1348/2689] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:12.916553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:12.918820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:12.997474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e3331-3831-4033-9db5-ad5fa3170f86 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:13.006030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:13.006829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:13.007177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:13.007714 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:13.019838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:13.021427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:13.037061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1342/2690] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:54:13.113568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce215b28-2651-4735-8fb9-ea1d346a42e7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:13.115172 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ce215b28-2651-4735-8fb9-ea1d346a42e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:13.191897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce215b28-2651-4735-8fb9-ea1d346a42e7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:13.196166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71883f-5f27-46e6-87a4-2b7342ff3478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:13.197533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71883f-5f27-46e6-87a4-2b7342ff3478 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:13.197533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71883f-5f27-46e6-87a4-2b7342ff3478 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:13.197533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71883f-5f27-46e6-87a4-2b7342ff3478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:13.200124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aa71883f-5f27-46e6-87a4-2b7342ff3478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:13.219869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:13.315543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71883f-5f27-46e6-87a4-2b7342ff3478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:13.317612 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71883f-5f27-46e6-87a4-2b7342ff3478 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:13.330274 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1349/2691] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:15.789392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:15.792038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:15.818621 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af939-8a49-4a6c-99e9-1ec5d9f795d0 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:15.828939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:15.830630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:15.831481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:15.832248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] PATH_INFO: `/v3/users/c0feb5a97c6a4e33861173c19b0d469f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:15.834774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0feb5a97c6a4e33861173c19b0d469f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:15.842164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:15.931535 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0feb5a97c6a4e33861173c19b0d469f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:15.932334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0feb5a97c6a4e33861173c19b0d469f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:15.946053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0feb5a97c6a4e33861173c19b0d469f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:15.946863 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0feb5a97c6a4e33861173c19b0d469f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:15.965487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0feb5a97c6a4e33861173c19b0d469f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:15.982008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1343/2692] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:15 2022] DELETE /identity/v3/users/c0feb5a97c6a4e33861173c19b0d469f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:15.987616 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56345fe-f5f5-4c97-a3c8-3d7347c34c74 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:15.988614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56345fe-f5f5-4c97-a3c8-3d7347c34c74 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:15.988614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56345fe-f5f5-4c97-a3c8-3d7347c34c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:16.010668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b56345fe-f5f5-4c97-a3c8-3d7347c34c74 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:16.012014 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b56345fe-f5f5-4c97-a3c8-3d7347c34c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:16.119060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1350/2693] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:16.140213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:16.147962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:16.247069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42736519-dcc9-4154-8eca-fe66bdb79bd6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:16.253735 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94264-7098-4762-ae96-4f77fd0b4580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:16.254777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94264-7098-4762-ae96-4f77fd0b4580 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:16.255224 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94264-7098-4762-ae96-4f77fd0b4580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:16.255695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94264-7098-4762-ae96-4f77fd0b4580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:16.272071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d94264-7098-4762-ae96-4f77fd0b4580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:16.273726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d94264-7098-4762-ae96-4f77fd0b4580 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:16.287217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1344/2694] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:16.906579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56345fe-f5f5-4c97-a3c8-3d7347c34c74 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:16.913839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b56345fe-f5f5-4c97-a3c8-3d7347c34c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:16.951992 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56345fe-f5f5-4c97-a3c8-3d7347c34c74 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:16.964794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:16.967802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:16.968483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:16.969028 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] PATH_INFO: `/v3/projects/63bc257821784396863250b764e6af8e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:16.972791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=63bc257821784396863250b764e6af8e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:16.978318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:17.123972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63bc257821784396863250b764e6af8e', 'request_id': 'req-a3caf66f-14d0-47f9-9019-1c6accdbc74a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:17.137799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63bc257821784396863250b764e6af8e', 'request_id': 'req-a3caf66f-14d0-47f9-9019-1c6accdbc74a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:17.146993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63bc257821784396863250b764e6af8e', 'request_id': 'req-a3caf66f-14d0-47f9-9019-1c6accdbc74a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:17.168386 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] The token cache is being invalidate because project 63bc257821784396863250b764e6af8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:17.169213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:17.171044 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a3caf66f-14d0-47f9-9019-1c6accdbc74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:17.173333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1351/2695] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:16 2022] DELETE /identity/v3/projects/63bc257821784396863250b764e6af8e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:17.180234 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7e1f2-e011-4265-b70e-bc0b50c2e9f5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:17.181207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7e1f2-e011-4265-b70e-bc0b50c2e9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:17.181700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7e1f2-e011-4265-b70e-bc0b50c2e9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:17.210019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:17.219087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e9a7e1f2-e011-4265-b70e-bc0b50c2e9f5 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:17.224383 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e9a7e1f2-e011-4265-b70e-bc0b50c2e9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:17.341792 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1345/2696] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:17.357535 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c09ef8-e385-4294-be7b-34a7053818c8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:17.357899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c09ef8-e385-4294-be7b-34a7053818c8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:17.358125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c09ef8-e385-4294-be7b-34a7053818c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:17.404823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c6c09ef8-e385-4294-be7b-34a7053818c8 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:17.408614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c6c09ef8-e385-4294-be7b-34a7053818c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:17.439438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1352/2697] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:17.443478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7e1f2-e011-4265-b70e-bc0b50c2e9f5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:17.458101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7e1f2-e011-4265-b70e-bc0b50c2e9f5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:17.473319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:17.474756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:17.475394 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:17.476049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:17.477473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:17.478183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:17.483306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:17.512769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1346/2698] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:17.518867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c09ef8-e385-4294-be7b-34a7053818c8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:17.538626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c09ef8-e385-4294-be7b-34a7053818c8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:17.548022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b7e76-1dfc-4fc6-aac2-d489e4545d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:17.549125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b7e76-1dfc-4fc6-aac2-d489e4545d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:17.549635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b7e76-1dfc-4fc6-aac2-d489e4545d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:17.550060 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b7e76-1dfc-4fc6-aac2-d489e4545d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:17.551263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b7e76-1dfc-4fc6-aac2-d489e4545d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:17.554975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b7e76-1dfc-4fc6-aac2-d489e4545d28 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:17.604350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:17.634164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1353/2699] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:17 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:17.635111 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:17.655460 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d449df-8ad3-44ec-9d4d-671cdbb18aaf admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:17.751051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ab112-961d-4d26-b03a-39dab0d327f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:17.751634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ab112-961d-4d26-b03a-39dab0d327f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:17.751907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ab112-961d-4d26-b03a-39dab0d327f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:17.752232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ab112-961d-4d26-b03a-39dab0d327f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:17.753057 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ab112-961d-4d26-b03a-39dab0d327f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:17.755711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ab112-961d-4d26-b03a-39dab0d327f8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:17.792008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1347/2700] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:17 2022] POST /identity/v3/users => generated 282 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:17.795004 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b7e76-1dfc-4fc6-aac2-d489e4545d28 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:17.812246 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b7e76-1dfc-4fc6-aac2-d489e4545d28 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:17.819701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07e72e-9b72-4af3-995c-0235563970aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:17.821293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07e72e-9b72-4af3-995c-0235563970aa admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:17.822023 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07e72e-9b72-4af3-995c-0235563970aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:17.822345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07e72e-9b72-4af3-995c-0235563970aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:17.823135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07e72e-9b72-4af3-995c-0235563970aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:17.826148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07e72e-9b72-4af3-995c-0235563970aa admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:17.846390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1354/2701] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:17.854728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ab112-961d-4d26-b03a-39dab0d327f8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:17.870915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ab112-961d-4d26-b03a-39dab0d327f8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:17.879537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176cf76-0003-46e6-80b3-05feb630e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:17.880667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176cf76-0003-46e6-80b3-05feb630e3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:17.881147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176cf76-0003-46e6-80b3-05feb630e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:17.882471 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176cf76-0003-46e6-80b3-05feb630e3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:17.883820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176cf76-0003-46e6-80b3-05feb630e3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:17.886627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176cf76-0003-46e6-80b3-05feb630e3d6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:17.922599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1348/2702] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:17.928964 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07e72e-9b72-4af3-995c-0235563970aa admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:17.947901 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07e72e-9b72-4af3-995c-0235563970aa admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:17.959384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc4b2-a5b3-4718-b7a0-040fb938b4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:17.960727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc4b2-a5b3-4718-b7a0-040fb938b4a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:17.961192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc4b2-a5b3-4718-b7a0-040fb938b4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:17.961651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc4b2-a5b3-4718-b7a0-040fb938b4a0 admin admin] PATH_INFO: `/v3/projects/ff3e306d69a949a5a5d320d41c0e0402/users/b9dea0f2eb0144fa9acea58f1372eac5/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:17.997035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fc4b2-a5b3-4718-b7a0-040fb938b4a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3e306d69a949a5a5d320d41c0e0402, user_id=b9dea0f2eb0144fa9acea58f1372eac5, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:18.002832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fc4b2-a5b3-4718-b7a0-040fb938b4a0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:18.063339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1355/2703] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:17 2022] PUT /identity/v3/projects/ff3e306d69a949a5a5d320d41c0e0402/users/b9dea0f2eb0144fa9acea58f1372eac5/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:18.067709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a09e-8735-40a9-bacd-b05a423d682a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:18.068220 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a09e-8735-40a9-bacd-b05a423d682a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:18.068703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a09e-8735-40a9-bacd-b05a423d682a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:18.148435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-58e3a09e-8735-40a9-bacd-b05a423d682a None None] MFA Rules not processed for user `b9dea0f2eb0144fa9acea58f1372eac5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:18.150110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-58e3a09e-8735-40a9-bacd-b05a423d682a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:18.304900 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1349/2704] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:18.311308 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc4b2-a5b3-4718-b7a0-040fb938b4a0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:18.466477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc4b2-a5b3-4718-b7a0-040fb938b4a0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:18.476102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271f100-38b7-428f-afe3-2e54b1a1765d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:18.476907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271f100-38b7-428f-afe3-2e54b1a1765d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:18.477253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271f100-38b7-428f-afe3-2e54b1a1765d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:18.482732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271f100-38b7-428f-afe3-2e54b1a1765d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:18.483226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271f100-38b7-428f-afe3-2e54b1a1765d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:18.485848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271f100-38b7-428f-afe3-2e54b1a1765d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:18.534199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1356/2705] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:18 2022] POST /identity/v3/projects => generated 365 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:18.534799 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3a09e-8735-40a9-bacd-b05a423d682a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:18.545274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:18.558305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3a09e-8735-40a9-bacd-b05a423d682a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:18.709010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70199a1-bd28-460e-9290-059dc2a39bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:18.710116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70199a1-bd28-460e-9290-059dc2a39bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:18.710445 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70199a1-bd28-460e-9290-059dc2a39bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:18.710921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70199a1-bd28-460e-9290-059dc2a39bdb admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:18.711912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70199a1-bd28-460e-9290-059dc2a39bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:18.714311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70199a1-bd28-460e-9290-059dc2a39bdb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:18.751295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1350/2706] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:18 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:18.753392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271f100-38b7-428f-afe3-2e54b1a1765d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:18.767437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271f100-38b7-428f-afe3-2e54b1a1765d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:18.775019 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d8f34-0399-48bb-bcca-c7fc08ee60d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:18.775980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d8f34-0399-48bb-bcca-c7fc08ee60d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:18.776319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d8f34-0399-48bb-bcca-c7fc08ee60d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:18.776720 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d8f34-0399-48bb-bcca-c7fc08ee60d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:18.778225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d8f34-0399-48bb-bcca-c7fc08ee60d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:18.779669 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d8f34-0399-48bb-bcca-c7fc08ee60d6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:18.797287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1357/2707] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:54:18.803369 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70199a1-bd28-460e-9290-059dc2a39bdb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:18.825326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70199a1-bd28-460e-9290-059dc2a39bdb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:18.845664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dde5c-5203-4214-8913-ea2008d50ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:18.846392 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dde5c-5203-4214-8913-ea2008d50ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:18.847069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dde5c-5203-4214-8913-ea2008d50ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:18.847289 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dde5c-5203-4214-8913-ea2008d50ac1 admin admin] PATH_INFO: `/v3/projects/8141e00faa1f4fa1a23f44ca470424d2/users/feb3d9a856b740f2984ad55a81020e37/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:18.901275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dde5c-5203-4214-8913-ea2008d50ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8141e00faa1f4fa1a23f44ca470424d2, user_id=feb3d9a856b740f2984ad55a81020e37, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:18.906602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dde5c-5203-4214-8913-ea2008d50ac1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:18.931544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1351/2708] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:18 2022] PUT /identity/v3/projects/8141e00faa1f4fa1a23f44ca470424d2/users/feb3d9a856b740f2984ad55a81020e37/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:18.937568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d8f34-0399-48bb-bcca-c7fc08ee60d6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:18.955809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d8f34-0399-48bb-bcca-c7fc08ee60d6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:19.049758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4763a-d057-4d2f-83d1-662ababb7f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:19.050547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4763a-d057-4d2f-83d1-662ababb7f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:19.051159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4763a-d057-4d2f-83d1-662ababb7f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:19.051497 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4763a-d057-4d2f-83d1-662ababb7f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:19.052503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4763a-d057-4d2f-83d1-662ababb7f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:19.054595 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4763a-d057-4d2f-83d1-662ababb7f54 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:19.074237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1358/2709] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:19.079794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dde5c-5203-4214-8913-ea2008d50ac1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:19.108782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dde5c-5203-4214-8913-ea2008d50ac1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:19.118448 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:19.119706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:19.121088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:19.121088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] PATH_INFO: `/v3/projects/8141e00faa1f4fa1a23f44ca470424d2/users/feb3d9a856b740f2984ad55a81020e37/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:19.124047 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8141e00faa1f4fa1a23f44ca470424d2, user_id=feb3d9a856b740f2984ad55a81020e37, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:19.129385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:19.198116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1352/2710] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:19 2022] PUT /identity/v3/projects/8141e00faa1f4fa1a23f44ca470424d2/users/feb3d9a856b740f2984ad55a81020e37/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:19.206906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5b8d7-0493-4a41-957b-467e1eeb0149 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:19.207676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5b8d7-0493-4a41-957b-467e1eeb0149 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:19.208042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5b8d7-0493-4a41-957b-467e1eeb0149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:19.208190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:19.210776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:19.330883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0ea5b8d7-0493-4a41-957b-467e1eeb0149 None None] MFA Rules not processed for user `feb3d9a856b740f2984ad55a81020e37`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:19.334189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0ea5b8d7-0493-4a41-957b-467e1eeb0149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:19.412185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:19.415685 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1b438-92cc-41e5-8081-afd1dedf35a0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:19.432358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:19.438678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:19.439212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:19.439744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] PATH_INFO: `/v3/users/bd03ff340761414885f054ccb2cd9eab` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:19.442610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd03ff340761414885f054ccb2cd9eab)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:19.448582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:19.533802 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1359/2711] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:19.556548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5b8d7-0493-4a41-957b-467e1eeb0149 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:19.559364 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0ea5b8d7-0493-4a41-957b-467e1eeb0149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:19.610001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd03ff340761414885f054ccb2cd9eab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:19.612451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd03ff340761414885f054ccb2cd9eab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:19.660580 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd03ff340761414885f054ccb2cd9eab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:19.662847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd03ff340761414885f054ccb2cd9eab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:19.673207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:19.711834 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4c9d35e9-4830-4376-a5e0-ebcfd3d22890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd03ff340761414885f054ccb2cd9eab'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:19.764464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1353/2712] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:19 2022] DELETE /identity/v3/users/bd03ff340761414885f054ccb2cd9eab => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:19.776249 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb42f65-fc09-4dab-b310-98e201b45f96 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:19.796470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb42f65-fc09-4dab-b310-98e201b45f96 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:19.797491 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb42f65-fc09-4dab-b310-98e201b45f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:19.888120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1fb42f65-fc09-4dab-b310-98e201b45f96 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:19.896544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1fb42f65-fc09-4dab-b310-98e201b45f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:20.108580 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5b8d7-0493-4a41-957b-467e1eeb0149 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:20.124319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1354/2713] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:20.142701 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb42f65-fc09-4dab-b310-98e201b45f96 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:20.146114 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1fb42f65-fc09-4dab-b310-98e201b45f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:20.272644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:20.273678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:20.274037 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:20.274501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:20.277056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:20.289484 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb42f65-fc09-4dab-b310-98e201b45f96 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:20.294911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad1e29-2f1e-49ab-936c-080d89a4ade5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:20.295869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad1e29-2f1e-49ab-936c-080d89a4ade5 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:20.296315 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad1e29-2f1e-49ab-936c-080d89a4ade5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:20.296760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad1e29-2f1e-49ab-936c-080d89a4ade5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:20.315067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad1e29-2f1e-49ab-936c-080d89a4ade5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:20.318619 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad1e29-2f1e-49ab-936c-080d89a4ade5 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:20.338067 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1355/2714] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:20.340021 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51285e8-38d2-411b-93e5-ddd5b5a3006d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:20.340413 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51285e8-38d2-411b-93e5-ddd5b5a3006d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:20.340858 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51285e8-38d2-411b-93e5-ddd5b5a3006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:20.391809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b51285e8-38d2-411b-93e5-ddd5b5a3006d None None] MFA Rules not processed for user `454a193b26684746bc8c9278a1f1f175`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:20.395473 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b51285e8-38d2-411b-93e5-ddd5b5a3006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:20.401975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:20.405008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:20.425231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1360/2715] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 870 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:54:20.580306 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1356/2716] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:20.610795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:20.615762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:20.715895 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:20.801852 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76ea4-b7d3-42f1-861a-ef91df8f3edd service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:20.813712 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31308976-1535-4458-9036-c74c23aafc86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:20.814118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31308976-1535-4458-9036-c74c23aafc86 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:20.814403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31308976-1535-4458-9036-c74c23aafc86 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:20.814822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31308976-1535-4458-9036-c74c23aafc86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:20.825034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31308976-1535-4458-9036-c74c23aafc86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:20.826754 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31308976-1535-4458-9036-c74c23aafc86 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:20.839950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1361/2717] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:54:20.890197 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51285e8-38d2-411b-93e5-ddd5b5a3006d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:20.898092 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b51285e8-38d2-411b-93e5-ddd5b5a3006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:20.933465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51285e8-38d2-411b-93e5-ddd5b5a3006d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:20.942956 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:20.944270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:20.944932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:20.945517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] PATH_INFO: `/v3/projects/03885b28674e445ebad72405a334f1f7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:20.947761 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=03885b28674e445ebad72405a334f1f7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:20.961525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:20.966677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:21.082588 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03885b28674e445ebad72405a334f1f7', 'request_id': 'req-b7149876-63d8-491d-94cb-f7cf0b48b74a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.096576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03885b28674e445ebad72405a334f1f7', 'request_id': 'req-b7149876-63d8-491d-94cb-f7cf0b48b74a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.116378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03885b28674e445ebad72405a334f1f7', 'request_id': 'req-b7149876-63d8-491d-94cb-f7cf0b48b74a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.140087 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] The token cache is being invalidate because project 03885b28674e445ebad72405a334f1f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:21.140618 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.141454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.144073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1357/2718] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:20 2022] DELETE /identity/v3/projects/03885b28674e445ebad72405a334f1f7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:21.149450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31308976-1535-4458-9036-c74c23aafc86 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:21.168780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31308976-1535-4458-9036-c74c23aafc86 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:21.297543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:21.299995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:21.300638 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:21.301112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] PATH_INFO: `/v3/users/3d7d9eb386c74be89f083cfa3e9c1f4c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:21.303987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7d9eb386c74be89f083cfa3e9c1f4c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:21.310492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:21.369336 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:21.372502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:21.420475 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7149876-63d8-491d-94cb-f7cf0b48b74a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:21.436521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:21.437671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:21.438189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:21.440031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] PATH_INFO: `/v3/users/b9dea0f2eb0144fa9acea58f1372eac5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:21.442747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9dea0f2eb0144fa9acea58f1372eac5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:21.447185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:21.469403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7d9eb386c74be89f083cfa3e9c1f4c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.469403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7d9eb386c74be89f083cfa3e9c1f4c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.504242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7d9eb386c74be89f083cfa3e9c1f4c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.534893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7d9eb386c74be89f083cfa3e9c1f4c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.559494 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9428a59a-4a4b-4c90-95be-3c503bf1be14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7d9eb386c74be89f083cfa3e9c1f4c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.575894 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1362/2719] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:21 2022] DELETE /identity/v3/users/3d7d9eb386c74be89f083cfa3e9c1f4c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:21.639388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9dea0f2eb0144fa9acea58f1372eac5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.641189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9dea0f2eb0144fa9acea58f1372eac5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.668652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9dea0f2eb0144fa9acea58f1372eac5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.669419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9dea0f2eb0144fa9acea58f1372eac5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.694023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9dea0f2eb0144fa9acea58f1372eac5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:21.727518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1358/2720] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:21 2022] DELETE /identity/v3/users/b9dea0f2eb0144fa9acea58f1372eac5 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:21.733875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25184-1062-46d8-81c9-cd5bcbc865c6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:21.735079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25184-1062-46d8-81c9-cd5bcbc865c6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:21.735079 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25184-1062-46d8-81c9-cd5bcbc865c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:21.773761 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a1d25184-1062-46d8-81c9-cd5bcbc865c6 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:21.776474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a1d25184-1062-46d8-81c9-cd5bcbc865c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:21.836147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:21.950635 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1363/2721] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:21.969593 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:21.972222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:22.067511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27aecf-99ab-44c6-9b64-aeb559bd6b65 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:22.073760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b5d2f-f1ca-445d-b522-c27629578d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:22.075842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5d2f-f1ca-445d-b522-c27629578d8e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:22.076174 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5d2f-f1ca-445d-b522-c27629578d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:22.076440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5d2f-f1ca-445d-b522-c27629578d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:22.099501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b5d2f-f1ca-445d-b522-c27629578d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:22.113020 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b5d2f-f1ca-445d-b522-c27629578d8e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:22.144543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1359/2722] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:22.253254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d25184-1062-46d8-81c9-cd5bcbc865c6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:22.255138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a1d25184-1062-46d8-81c9-cd5bcbc865c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:22.268816 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d25184-1062-46d8-81c9-cd5bcbc865c6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:22.275194 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:22.276285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:22.276768 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:22.277268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] PATH_INFO: `/v3/projects/2a243587231b46a1a92960f94100f32c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:22.278789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a243587231b46a1a92960f94100f32c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:22.281114 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:22.394396 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a243587231b46a1a92960f94100f32c', 'request_id': 'req-92f410a8-1475-438b-a4e7-5573a5384556'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.417228 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a243587231b46a1a92960f94100f32c', 'request_id': 'req-92f410a8-1475-438b-a4e7-5573a5384556'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.441201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a243587231b46a1a92960f94100f32c', 'request_id': 'req-92f410a8-1475-438b-a4e7-5573a5384556'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.480826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] The token cache is being invalidate because project 2a243587231b46a1a92960f94100f32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:22.482643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.485223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-92f410a8-1475-438b-a4e7-5573a5384556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.489585 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1364/2723] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:22 2022] DELETE /identity/v3/projects/2a243587231b46a1a92960f94100f32c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:22.499526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5342b6b-1af9-4c73-b60e-ae8916ee6e25 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:22.500251 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5342b6b-1af9-4c73-b60e-ae8916ee6e25 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:22.500793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5342b6b-1af9-4c73-b60e-ae8916ee6e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:22.520076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:22.536283 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f5342b6b-1af9-4c73-b60e-ae8916ee6e25 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:22.538348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f5342b6b-1af9-4c73-b60e-ae8916ee6e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:22.616783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1360/2724] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:22.627415 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a58da-6bd6-4b10-b330-1e0537888182 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:22.629349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a58da-6bd6-4b10-b330-1e0537888182 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:22.630480 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a58da-6bd6-4b10-b330-1e0537888182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:22.666821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5a2a58da-6bd6-4b10-b330-1e0537888182 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:22.669140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a2a58da-6bd6-4b10-b330-1e0537888182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:22.698339 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1365/2725] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:22.699662 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5342b6b-1af9-4c73-b60e-ae8916ee6e25 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:22.710763 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5342b6b-1af9-4c73-b60e-ae8916ee6e25 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:22.717227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:22.718040 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:22.718378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:22.718670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:22.719449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:22.719826 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:22.722408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:22.737254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a58da-6bd6-4b10-b330-1e0537888182 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:22.738688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a2a58da-6bd6-4b10-b330-1e0537888182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:22.748983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1361/2726] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:22.754360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:22.762348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a58da-6bd6-4b10-b330-1e0537888182 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:22.773478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225ffc6-5490-46e8-bd20-d2039374d1bc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:22.780402 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9492783-61cb-48d6-beea-e7096f168355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:22.781210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9492783-61cb-48d6-beea-e7096f168355 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:22.781613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:22.781786 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9492783-61cb-48d6-beea-e7096f168355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:22.782110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9492783-61cb-48d6-beea-e7096f168355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:22.782952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9492783-61cb-48d6-beea-e7096f168355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:22.784065 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:22.784982 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:22.785717 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] PATH_INFO: `/v3/projects/ff3e306d69a949a5a5d320d41c0e0402` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:22.786009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9492783-61cb-48d6-beea-e7096f168355 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:22.788610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3e306d69a949a5a5d320d41c0e0402)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:22.793884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:22.834238 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1362/2727] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:22 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:22.839166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9492783-61cb-48d6-beea-e7096f168355 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:22.929273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3e306d69a949a5a5d320d41c0e0402', 'request_id': 'req-80ffd9e0-9920-4ef7-b646-320dad22963f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.944440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3e306d69a949a5a5d320d41c0e0402', 'request_id': 'req-80ffd9e0-9920-4ef7-b646-320dad22963f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.959515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3e306d69a949a5a5d320d41c0e0402', 'request_id': 'req-80ffd9e0-9920-4ef7-b646-320dad22963f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.994078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] The token cache is being invalidate because project ff3e306d69a949a5a5d320d41c0e0402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:22.994525 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.995571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:22.997764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1366/2728] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:22 2022] DELETE /identity/v3/projects/ff3e306d69a949a5a5d320d41c0e0402 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:23.003825 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:23.005538 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:23.008032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9492783-61cb-48d6-beea-e7096f168355 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:23.073075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:23.120168 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:23.121268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:23.121750 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:23.122636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:23.123980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:23.128479 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:23.157094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffd9e0-9920-4ef7-b646-320dad22963f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:23.167757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:23.169044 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:23.169529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:23.169977 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] PATH_INFO: `/v3/users/feb3d9a856b740f2984ad55a81020e37` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:23.171951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb3d9a856b740f2984ad55a81020e37)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:23.176142 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:23.180077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1363/2729] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:22 2022] POST /identity/v3/users => generated 288 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:23.183143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:23.188894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:23.215444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e0c9c-d8f5-4b67-af18-ccccc10ce34c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:23.226166 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7ec65-4c0c-4ae6-b473-75a4ba907179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:23.227325 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7ec65-4c0c-4ae6-b473-75a4ba907179 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:23.227623 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7ec65-4c0c-4ae6-b473-75a4ba907179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:23.228412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7ec65-4c0c-4ae6-b473-75a4ba907179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:23.230060 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7ec65-4c0c-4ae6-b473-75a4ba907179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:23.234349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7ec65-4c0c-4ae6-b473-75a4ba907179 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:23.237377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:23.265541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1364/2730] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:23.272136 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7ec65-4c0c-4ae6-b473-75a4ba907179 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:23.284727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7ec65-4c0c-4ae6-b473-75a4ba907179 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:23.296414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767623b7-b08e-49b5-81c2-fab9d053552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:23.297222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767623b7-b08e-49b5-81c2-fab9d053552b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:23.297596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767623b7-b08e-49b5-81c2-fab9d053552b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:23.298076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767623b7-b08e-49b5-81c2-fab9d053552b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:23.299225 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767623b7-b08e-49b5-81c2-fab9d053552b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:23.303045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767623b7-b08e-49b5-81c2-fab9d053552b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:23.308348 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb3d9a856b740f2984ad55a81020e37'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:23.309800 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb3d9a856b740f2984ad55a81020e37'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:23.327840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1365/2731] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:54:23.329729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767623b7-b08e-49b5-81c2-fab9d053552b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:23.343617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb3d9a856b740f2984ad55a81020e37'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:23.345684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767623b7-b08e-49b5-81c2-fab9d053552b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:23.354470 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a74481-af00-4edc-839f-a0921405271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:23.355360 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a74481-af00-4edc-839f-a0921405271c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:23.355738 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a74481-af00-4edc-839f-a0921405271c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:23.356276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a74481-af00-4edc-839f-a0921405271c admin admin] PATH_INFO: `/v3/projects/513f5ad5cf9c497d987e48d92bafa0ad/users/c8a1d291e6be4785b9178e7db6c0056c/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:23.366489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb3d9a856b740f2984ad55a81020e37'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:23.389477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5ebae523-6c8c-4ab7-8add-553aff7e276b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb3d9a856b740f2984ad55a81020e37'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:23.391085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a74481-af00-4edc-839f-a0921405271c admin admin] RBAC: Authorizing `identity:create_grant(project_id=513f5ad5cf9c497d987e48d92bafa0ad, user_id=c8a1d291e6be4785b9178e7db6c0056c, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:23.395404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a74481-af00-4edc-839f-a0921405271c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:23.416608 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1367/2732] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:23 2022] DELETE /identity/v3/users/feb3d9a856b740f2984ad55a81020e37 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:23.441699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1366/2733] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:23 2022] PUT /identity/v3/projects/513f5ad5cf9c497d987e48d92bafa0ad/users/c8a1d291e6be4785b9178e7db6c0056c/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:23.457027 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd09da4-578f-45c5-905b-798905494133 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:23.457829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd09da4-578f-45c5-905b-798905494133 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:23.458731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd09da4-578f-45c5-905b-798905494133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:23.504468 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a7e2-252f-4c9c-b415-7eb7d437fac7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:23.505063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a7e2-252f-4c9c-b415-7eb7d437fac7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:23.505486 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a7e2-252f-4c9c-b415-7eb7d437fac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:23.540431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0250a7e2-252f-4c9c-b415-7eb7d437fac7 None None] MFA Rules not processed for user `454a193b26684746bc8c9278a1f1f175`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:23.543511 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0250a7e2-252f-4c9c-b415-7eb7d437fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:23.559407 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3cd09da4-578f-45c5-905b-798905494133 None None] MFA Rules not processed for user `c8a1d291e6be4785b9178e7db6c0056c`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:23.562018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3cd09da4-578f-45c5-905b-798905494133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:23.672418 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1367/2734] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:23.723183 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1368/2735] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:23.733754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250a7e2-252f-4c9c-b415-7eb7d437fac7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:23.736812 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0250a7e2-252f-4c9c-b415-7eb7d437fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:23.828296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250a7e2-252f-4c9c-b415-7eb7d437fac7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:23.834977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8393c-4dca-46e8-b368-77434faba093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:23.835756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8393c-4dca-46e8-b368-77434faba093 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:23.836104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8393c-4dca-46e8-b368-77434faba093 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:23.836551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8393c-4dca-46e8-b368-77434faba093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:23.850562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8393c-4dca-46e8-b368-77434faba093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:23.852436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8393c-4dca-46e8-b368-77434faba093 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:23.867762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1368/2736] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:23.953656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd09da4-578f-45c5-905b-798905494133 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:23.956220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3cd09da4-578f-45c5-905b-798905494133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:24.027056 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:24.080537 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd09da4-578f-45c5-905b-798905494133 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:24.099842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:24.100911 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:24.101423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:24.101938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] PATH_INFO: `/v3/projects/8141e00faa1f4fa1a23f44ca470424d2` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:24.103615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8141e00faa1f4fa1a23f44ca470424d2)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:24.105925 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:24.225997 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8141e00faa1f4fa1a23f44ca470424d2', 'request_id': 'req-42e00e34-a329-4f36-ab34-e1d1734b67b1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:24.244362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8141e00faa1f4fa1a23f44ca470424d2', 'request_id': 'req-42e00e34-a329-4f36-ab34-e1d1734b67b1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:24.258034 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8141e00faa1f4fa1a23f44ca470424d2', 'request_id': 'req-42e00e34-a329-4f36-ab34-e1d1734b67b1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:24.283890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] The token cache is being invalidate because project 8141e00faa1f4fa1a23f44ca470424d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:24.284388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:24.285502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:24.287731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1369/2737] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:23 2022] DELETE /identity/v3/projects/8141e00faa1f4fa1a23f44ca470424d2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:24.294311 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ad99-ad18-4da4-9451-c6f33ef40156 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:24.294582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ad99-ad18-4da4-9451-c6f33ef40156 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:24.295275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ad99-ad18-4da4-9451-c6f33ef40156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:24.333554 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-76f5ad99-ad18-4da4-9451-c6f33ef40156 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:24.335647 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-76f5ad99-ad18-4da4-9451-c6f33ef40156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:24.348459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:24.350937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:24.478125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1369/2738] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:24.488065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fbe5a-3ebe-417f-8288-404b04754a6f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:24.488313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fbe5a-3ebe-417f-8288-404b04754a6f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:24.488487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fbe5a-3ebe-417f-8288-404b04754a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:24.500671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e00e34-a329-4f36-ab34-e1d1734b67b1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:24.506319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:24.507446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:24.507906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:24.508463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:24.510883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:24.525463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:24.527033 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0c8fbe5a-3ebe-417f-8288-404b04754a6f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:24.528389 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:24.528992 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0c8fbe5a-3ebe-417f-8288-404b04754a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:24.542998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1370/2739] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:24.543606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1370/2740] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:24.544622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:24.559751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6208-2a44-40e5-8670-76b15e949a2e service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:24.568725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:24.570182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:24.570783 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:24.571424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:24.572632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:24.573466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:24.577281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:24.602636 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1371/2741] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:24.607463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fbe5a-3ebe-417f-8288-404b04754a6f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:24.613401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:24.637204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fbe5a-3ebe-417f-8288-404b04754a6f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:24.644672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f6b4f-fa15-4afb-a3e0-d6216c114934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:24.645612 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6b4f-fa15-4afb-a3e0-d6216c114934 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:24.646278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6b4f-fa15-4afb-a3e0-d6216c114934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:24.646700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6b4f-fa15-4afb-a3e0-d6216c114934 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:24.650540 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f6b4f-fa15-4afb-a3e0-d6216c114934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:24.652879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f6b4f-fa15-4afb-a3e0-d6216c114934 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:24.716917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1371/2742] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:24 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:24.718138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:24.740112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d83a5-d22e-4e18-870f-c678e9ea1aad admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:24.896627 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9dedd-c14e-4448-bb72-c81c278d19e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:24.897552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9dedd-c14e-4448-bb72-c81c278d19e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:24.897552 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9dedd-c14e-4448-bb72-c81c278d19e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:24.897784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9dedd-c14e-4448-bb72-c81c278d19e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:24.898416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9dedd-c14e-4448-bb72-c81c278d19e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:24.900630 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9dedd-c14e-4448-bb72-c81c278d19e9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:24.910081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:24.933320 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1372/2743] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:24 2022] POST /identity/v3/users => generated 291 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:24.939444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f6b4f-fa15-4afb-a3e0-d6216c114934 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:24.957909 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f6b4f-fa15-4afb-a3e0-d6216c114934 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:24.967490 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1680d4-064c-4494-894d-252aa8428e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:24.968599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1680d4-064c-4494-894d-252aa8428e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:24.969052 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1680d4-064c-4494-894d-252aa8428e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:24.969533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1680d4-064c-4494-894d-252aa8428e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:24.971294 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1680d4-064c-4494-894d-252aa8428e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:24.975485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1680d4-064c-4494-894d-252aa8428e2b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:24.996794 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1372/2744] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:25.004854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9dedd-c14e-4448-bb72-c81c278d19e9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:25.026701 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9dedd-c14e-4448-bb72-c81c278d19e9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:25.032516 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:25.033141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:25.033805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:25.034818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:25.035672 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:25.037363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:25.056115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1373/2745] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:25.060069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1680d4-064c-4494-894d-252aa8428e2b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:25.088639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1680d4-064c-4494-894d-252aa8428e2b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:25.103137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a1a04-282f-4b4d-a2bd-a189d39b5f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:25.104099 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a1a04-282f-4b4d-a2bd-a189d39b5f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:25.104477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a1a04-282f-4b4d-a2bd-a189d39b5f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:25.105011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a1a04-282f-4b4d-a2bd-a189d39b5f59 admin admin] PATH_INFO: `/v3/projects/b0a7b05d189e4977bd88e750f1121ab2/users/348959b074954f5e941f2796fbcc56e2/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:25.113705 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:25.116670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:25.126424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a1a04-282f-4b4d-a2bd-a189d39b5f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a7b05d189e4977bd88e750f1121ab2, user_id=348959b074954f5e941f2796fbcc56e2, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:25.130065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a1a04-282f-4b4d-a2bd-a189d39b5f59 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:25.155665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1373/2746] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:25 2022] PUT /identity/v3/projects/b0a7b05d189e4977bd88e750f1121ab2/users/348959b074954f5e941f2796fbcc56e2/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:25.172798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd45e94-38d0-479b-ba60-b2a494bda1ba None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:25.173063 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd45e94-38d0-479b-ba60-b2a494bda1ba None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:25.173346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd45e94-38d0-479b-ba60-b2a494bda1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:25.231712 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-acd45e94-38d0-479b-ba60-b2a494bda1ba None None] MFA Rules not processed for user `348959b074954f5e941f2796fbcc56e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:25.233210 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-acd45e94-38d0-479b-ba60-b2a494bda1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:25.276025 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db985f6-15d8-4e29-8292-212d2d3ca9e6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:25.375522 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1374/2747] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:25.381981 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd45e94-38d0-479b-ba60-b2a494bda1ba None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:25.411466 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:25.413238 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:25.413609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:25.414047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:25.416995 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:25.434811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:25.436677 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:25.464022 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1374/2748] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:25.518237 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd45e94-38d0-479b-ba60-b2a494bda1ba None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:25.528141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4482ff5-286e-4ba2-b298-f9cc30017b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:25.529335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482ff5-286e-4ba2-b298-f9cc30017b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:25.529918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482ff5-286e-4ba2-b298-f9cc30017b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:25.530500 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482ff5-286e-4ba2-b298-f9cc30017b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:25.532442 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4482ff5-286e-4ba2-b298-f9cc30017b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:25.537537 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4482ff5-286e-4ba2-b298-f9cc30017b9e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:25.616790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1375/2749] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:25 2022] POST /identity/v3/projects => generated 381 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:25.620013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:25.634384 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e098c-adda-4b48-a2be-b852e4914b54 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:25.775152 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed1e9-1e83-4d2f-a2b9-04babb275860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:25.777141 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed1e9-1e83-4d2f-a2b9-04babb275860 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:25.777734 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed1e9-1e83-4d2f-a2b9-04babb275860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:25.778296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed1e9-1e83-4d2f-a2b9-04babb275860 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:25.779523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ed1e9-1e83-4d2f-a2b9-04babb275860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:25.785622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ed1e9-1e83-4d2f-a2b9-04babb275860 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:25.829453 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:25.865292 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1375/2750] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:25 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:25.872575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4482ff5-286e-4ba2-b298-f9cc30017b9e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:25.893899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4482ff5-286e-4ba2-b298-f9cc30017b9e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:25.905403 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c416233-dd01-4ab9-ad2f-c6a2f1f4152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:25.908124 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416233-dd01-4ab9-ad2f-c6a2f1f4152a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:25.908711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416233-dd01-4ab9-ad2f-c6a2f1f4152a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:25.909199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416233-dd01-4ab9-ad2f-c6a2f1f4152a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:25.917228 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c416233-dd01-4ab9-ad2f-c6a2f1f4152a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:25.919816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c416233-dd01-4ab9-ad2f-c6a2f1f4152a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:25.954669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1376/2751] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:25.955799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed1e9-1e83-4d2f-a2b9-04babb275860 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:25.969548 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed1e9-1e83-4d2f-a2b9-04babb275860 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:25.978819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72849a-a7bc-4ccc-906c-c8caf1035b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:25.979561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72849a-a7bc-4ccc-906c-c8caf1035b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:25.979899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72849a-a7bc-4ccc-906c-c8caf1035b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:25.980177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72849a-a7bc-4ccc-906c-c8caf1035b92 admin admin] PATH_INFO: `/v3/projects/d7256a7539d141639d13ed744ddc4b0f/users/70e7313ccbae4d4388da44fb7fba0294/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:26.014655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72849a-a7bc-4ccc-906c-c8caf1035b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7256a7539d141639d13ed744ddc4b0f, user_id=70e7313ccbae4d4388da44fb7fba0294, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:26.024660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72849a-a7bc-4ccc-906c-c8caf1035b92 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:26.056470 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1376/2752] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:25 2022] PUT /identity/v3/projects/d7256a7539d141639d13ed744ddc4b0f/users/70e7313ccbae4d4388da44fb7fba0294/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:26.062726 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c416233-dd01-4ab9-ad2f-c6a2f1f4152a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:26.086844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c416233-dd01-4ab9-ad2f-c6a2f1f4152a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:26.101036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:26.182902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786402ec-7848-424b-be09-bf729e982b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:26.184172 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786402ec-7848-424b-be09-bf729e982b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:26.184617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786402ec-7848-424b-be09-bf729e982b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:26.185059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786402ec-7848-424b-be09-bf729e982b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:26.186056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786402ec-7848-424b-be09-bf729e982b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:26.188902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786402ec-7848-424b-be09-bf729e982b1b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:26.205010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1377/2753] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:26.211097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72849a-a7bc-4ccc-906c-c8caf1035b92 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:26.228344 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72849a-a7bc-4ccc-906c-c8caf1035b92 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:26.233733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:26.234624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:26.235018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:26.235565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] PATH_INFO: `/v3/projects/d7256a7539d141639d13ed744ddc4b0f/users/70e7313ccbae4d4388da44fb7fba0294/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:26.240009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7256a7539d141639d13ed744ddc4b0f, user_id=70e7313ccbae4d4388da44fb7fba0294, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:26.245613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:26.267345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1377/2754] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:26 2022] PUT /identity/v3/projects/d7256a7539d141639d13ed744ddc4b0f/users/70e7313ccbae4d4388da44fb7fba0294/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:26.370175 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acd277-8a7b-4a7a-99fe-1af18a272b37 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:26.370443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acd277-8a7b-4a7a-99fe-1af18a272b37 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:26.370743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acd277-8a7b-4a7a-99fe-1af18a272b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:26.406703 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b3acd277-8a7b-4a7a-99fe-1af18a272b37 None None] MFA Rules not processed for user `70e7313ccbae4d4388da44fb7fba0294`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:26.408164 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b3acd277-8a7b-4a7a-99fe-1af18a272b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:26.495066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1378/2755] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:26.512307 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:26.514622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:26.590038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3b165-32ad-4078-a966-c82fbe1c29cd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:26.594637 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914b91b-2058-4a43-a7c8-0883ddf1ee42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:26.595657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914b91b-2058-4a43-a7c8-0883ddf1ee42 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:26.596013 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914b91b-2058-4a43-a7c8-0883ddf1ee42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:26.596315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914b91b-2058-4a43-a7c8-0883ddf1ee42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:26.607147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914b91b-2058-4a43-a7c8-0883ddf1ee42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:26.608670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914b91b-2058-4a43-a7c8-0883ddf1ee42 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:26.620160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1378/2756] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:27.100169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acd277-8a7b-4a7a-99fe-1af18a272b37 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:27.111356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acd277-8a7b-4a7a-99fe-1af18a272b37 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:27.118670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e7313ccbae4d4388da44fb7fba0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7256a7539d141639d13ed744ddc4b0f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:27.119604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:27.119936 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:27.120267 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:27.121097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:27.121453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:27.123204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:27.139273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1379/2757] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:27.145827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914b91b-2058-4a43-a7c8-0883ddf1ee42 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:27.163008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914b91b-2058-4a43-a7c8-0883ddf1ee42 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:27.171372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a326-0aa2-4d70-9ab3-6b23b04a8e26 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e7313ccbae4d4388da44fb7fba0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7256a7539d141639d13ed744ddc4b0f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:27.172401 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a326-0aa2-4d70-9ab3-6b23b04a8e26 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:27.172900 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a326-0aa2-4d70-9ab3-6b23b04a8e26 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:27.173392 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a326-0aa2-4d70-9ab3-6b23b04a8e26 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:27.174610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4a326-0aa2-4d70-9ab3-6b23b04a8e26 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:27.178217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4a326-0aa2-4d70-9ab3-6b23b04a8e26 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:27.224733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1379/2758] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:27 2022] POST /identity/v3/projects => generated 352 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:27.228389 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:27.247578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8089aa5-4e0d-4cce-b79a-be00c845874b tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:27.354415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e7313ccbae4d4388da44fb7fba0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7256a7539d141639d13ed744ddc4b0f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:27.355273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:27.355690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:27.356008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:27.357094 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:27.357457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:27.359393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:27.377919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1380/2759] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:54:27.443372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a326-0aa2-4d70-9ab3-6b23b04a8e26 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:27.460610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a326-0aa2-4d70-9ab3-6b23b04a8e26 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:27.469849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e7313ccbae4d4388da44fb7fba0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7256a7539d141639d13ed744ddc4b0f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:27.470972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:27.471504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:27.472006 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] PATH_INFO: `/v3/projects/8dfa4e5d2e924dc5822b174899ce922d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:27.474346 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8dfa4e5d2e924dc5822b174899ce922d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:27.478263 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:27.559540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dfa4e5d2e924dc5822b174899ce922d', 'request_id': 'req-ed452320-b220-466e-91dc-3998f17d6647'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:27.570101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dfa4e5d2e924dc5822b174899ce922d', 'request_id': 'req-ed452320-b220-466e-91dc-3998f17d6647'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:27.584799 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dfa4e5d2e924dc5822b174899ce922d', 'request_id': 'req-ed452320-b220-466e-91dc-3998f17d6647'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:27.609541 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] The token cache is being invalidate because project 8dfa4e5d2e924dc5822b174899ce922d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:27.610032 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:27.611113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed452320-b220-466e-91dc-3998f17d6647 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:27.614116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1380/2760] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:27 2022] DELETE /identity/v3/projects/8dfa4e5d2e924dc5822b174899ce922d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:27.871349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:27.873583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:27.894345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:27.986436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ed71-b0af-4b75-9357-9906adb6c265 tempest-VolumeQuotaClassesTest-1969021046 tempest-VolumeQuotaClassesTest-1969021046-project-admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:28.060877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:28.061441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:28.061723 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:28.061999 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] PATH_INFO: `/v3/users/348959b074954f5e941f2796fbcc56e2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:28.063395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348959b074954f5e941f2796fbcc56e2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:28.066048 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:28.129821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348959b074954f5e941f2796fbcc56e2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.131769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348959b074954f5e941f2796fbcc56e2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.143116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348959b074954f5e941f2796fbcc56e2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.144231 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348959b074954f5e941f2796fbcc56e2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.160480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348959b074954f5e941f2796fbcc56e2'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.174291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1381/2761] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:27 2022] DELETE /identity/v3/users/348959b074954f5e941f2796fbcc56e2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:28.180487 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435a7a9-c62a-41de-8179-7793884e5c40 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:28.181087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435a7a9-c62a-41de-8179-7793884e5c40 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:28.181087 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435a7a9-c62a-41de-8179-7793884e5c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:28.202204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8435a7a9-c62a-41de-8179-7793884e5c40 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:28.203511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8435a7a9-c62a-41de-8179-7793884e5c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:28.229780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:28.307129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1381/2762] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:28.318388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:28.320000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:28.372026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:28.398985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761cdb8-b7d4-4b5e-9bfc-3abd35f44e92 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:28.403498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf6ee2-0e89-4a8d-ad29-8d59216f7830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:28.404230 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6ee2-0e89-4a8d-ad29-8d59216f7830 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:28.404512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6ee2-0e89-4a8d-ad29-8d59216f7830 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:28.404779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6ee2-0e89-4a8d-ad29-8d59216f7830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:28.416752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf6ee2-0e89-4a8d-ad29-8d59216f7830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:28.419378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf6ee2-0e89-4a8d-ad29-8d59216f7830 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:28.437884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1382/2763] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:28.824543 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435a7a9-c62a-41de-8179-7793884e5c40 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:28.826776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8435a7a9-c62a-41de-8179-7793884e5c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:28.854178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435a7a9-c62a-41de-8179-7793884e5c40 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:28.863388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:28.864474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:28.864921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:28.865388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] PATH_INFO: `/v3/projects/b0a7b05d189e4977bd88e750f1121ab2` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:28.867807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0a7b05d189e4977bd88e750f1121ab2)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:28.871460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:28.942069 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0a7b05d189e4977bd88e750f1121ab2', 'request_id': 'req-3e69a601-9b59-4921-974f-b6716d59cdf7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.948474 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0a7b05d189e4977bd88e750f1121ab2', 'request_id': 'req-3e69a601-9b59-4921-974f-b6716d59cdf7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.955632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0a7b05d189e4977bd88e750f1121ab2', 'request_id': 'req-3e69a601-9b59-4921-974f-b6716d59cdf7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.976540 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] The token cache is being invalidate because project b0a7b05d189e4977bd88e750f1121ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:28.977049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.978158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:28.980688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1382/2764] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:28 2022] DELETE /identity/v3/projects/b0a7b05d189e4977bd88e750f1121ab2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:28.986700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf6ee2-0e89-4a8d-ad29-8d59216f7830 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:28.998546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf6ee2-0e89-4a8d-ad29-8d59216f7830 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:29.085186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:29.085904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:29.086182 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:29.086458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] PATH_INFO: `/v3/users/70e7313ccbae4d4388da44fb7fba0294` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:29.087917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=70e7313ccbae4d4388da44fb7fba0294)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:29.090004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:29.163581 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e7313ccbae4d4388da44fb7fba0294'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.165472 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e7313ccbae4d4388da44fb7fba0294'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.180115 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e7313ccbae4d4388da44fb7fba0294'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.181254 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e7313ccbae4d4388da44fb7fba0294'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.203658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-068da2e2-9c85-4dc2-a43b-05241b9e54ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e7313ccbae4d4388da44fb7fba0294'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.226278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1383/2765] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:28 2022] DELETE /identity/v3/users/70e7313ccbae4d4388da44fb7fba0294 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:29.600780 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:29.604045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:29.632814 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69a601-9b59-4921-974f-b6716d59cdf7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:29.767765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:29.768951 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:29.769452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:29.769955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] PATH_INFO: `/v3/projects/d7256a7539d141639d13ed744ddc4b0f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:29.772192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7256a7539d141639d13ed744ddc4b0f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:29.776596 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:29.918103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7256a7539d141639d13ed744ddc4b0f', 'request_id': 'req-09f39450-98c1-489e-b1a6-da9f96c0ef09'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.922615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:29.932889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7256a7539d141639d13ed744ddc4b0f', 'request_id': 'req-09f39450-98c1-489e-b1a6-da9f96c0ef09'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.948325 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7256a7539d141639d13ed744ddc4b0f', 'request_id': 'req-09f39450-98c1-489e-b1a6-da9f96c0ef09'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.976352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] The token cache is being invalidate because project d7256a7539d141639d13ed744ddc4b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:29.976870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.978153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-09f39450-98c1-489e-b1a6-da9f96c0ef09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:29.980822 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1383/2766] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:29 2022] DELETE /identity/v3/projects/d7256a7539d141639d13ed744ddc4b0f => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:29.988589 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa9199-1d35-4c4d-a8d5-a5ee27817a03 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:29.989085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa9199-1d35-4c4d-a8d5-a5ee27817a03 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:29.989575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa9199-1d35-4c4d-a8d5-a5ee27817a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.021690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e2fa9199-1d35-4c4d-a8d5-a5ee27817a03 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:30.022937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e2fa9199-1d35-4c4d-a8d5-a5ee27817a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:30.091213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:30.117886 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1384/2767] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:30.126287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106a889-1105-4448-9efc-737c9fcc6fab None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:30.126748 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106a889-1105-4448-9efc-737c9fcc6fab None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:30.127276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106a889-1105-4448-9efc-737c9fcc6fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.154378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9106a889-1105-4448-9efc-737c9fcc6fab None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:30.155682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9106a889-1105-4448-9efc-737c9fcc6fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:30.170385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1384/2768] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:30.176154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa9199-1d35-4c4d-a8d5-a5ee27817a03 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:30.192867 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa9199-1d35-4c4d-a8d5-a5ee27817a03 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:30.206443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:30.207686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:30.207995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:30.208388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.210295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:30.210466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:30.212065 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:30.232645 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1385/2769] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:30.237917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106a889-1105-4448-9efc-737c9fcc6fab None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:30.256658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106a889-1105-4448-9efc-737c9fcc6fab None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:30.264260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261245-0ac3-4dcf-ae63-3b0071c3589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:30.265310 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261245-0ac3-4dcf-ae63-3b0071c3589a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:30.265749 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261245-0ac3-4dcf-ae63-3b0071c3589a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:30.266150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261245-0ac3-4dcf-ae63-3b0071c3589a admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.267418 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b261245-0ac3-4dcf-ae63-3b0071c3589a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:30.271278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b261245-0ac3-4dcf-ae63-3b0071c3589a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:30.326296 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1385/2770] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:30 2022] POST /identity/v3/projects => generated 389 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:30.327674 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:30.345487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160b01c-44e8-4430-ab0c-28b978fc968a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:30.463526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b03c-2210-4899-9ffa-75a54e04d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:30.464513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b03c-2210-4899-9ffa-75a54e04d26f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:30.464964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b03c-2210-4899-9ffa-75a54e04d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:30.465442 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b03c-2210-4899-9ffa-75a54e04d26f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.466592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86b03c-2210-4899-9ffa-75a54e04d26f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:30.470316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86b03c-2210-4899-9ffa-75a54e04d26f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:30.508217 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1386/2771] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:30 2022] POST /identity/v3/users => generated 295 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:30.515936 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261245-0ac3-4dcf-ae63-3b0071c3589a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:30.535371 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261245-0ac3-4dcf-ae63-3b0071c3589a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:30.543953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068452e1-b4d5-45b0-a437-aab412b1ea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:30.545153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068452e1-b4d5-45b0-a437-aab412b1ea31 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:30.545718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068452e1-b4d5-45b0-a437-aab412b1ea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:30.546276 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068452e1-b4d5-45b0-a437-aab412b1ea31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.547545 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068452e1-b4d5-45b0-a437-aab412b1ea31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:30.551297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068452e1-b4d5-45b0-a437-aab412b1ea31 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:30.575315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1386/2772] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:30.576753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b03c-2210-4899-9ffa-75a54e04d26f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:30.593973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b03c-2210-4899-9ffa-75a54e04d26f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:30.603559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b393a2b-7de8-4be8-a002-5465c9da6a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:30.604609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b393a2b-7de8-4be8-a002-5465c9da6a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:30.605053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b393a2b-7de8-4be8-a002-5465c9da6a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:30.605495 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b393a2b-7de8-4be8-a002-5465c9da6a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.606709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b393a2b-7de8-4be8-a002-5465c9da6a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:30.610505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b393a2b-7de8-4be8-a002-5465c9da6a9b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:30.635368 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1387/2773] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:30.641143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068452e1-b4d5-45b0-a437-aab412b1ea31 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:30.658421 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068452e1-b4d5-45b0-a437-aab412b1ea31 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:30.666934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ee744-cdeb-4d47-a68f-384bb9748749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:30.668067 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ee744-cdeb-4d47-a68f-384bb9748749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:30.668562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ee744-cdeb-4d47-a68f-384bb9748749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:30.668966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ee744-cdeb-4d47-a68f-384bb9748749 admin admin] PATH_INFO: `/v3/projects/51e1e1403bab4c50820547962b347992/users/ada68d81b7684081b5bf220290936d89/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.696829 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ee744-cdeb-4d47-a68f-384bb9748749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e1e1403bab4c50820547962b347992, user_id=ada68d81b7684081b5bf220290936d89, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:30.703085 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ee744-cdeb-4d47-a68f-384bb9748749 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:30.743343 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1387/2774] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:30 2022] PUT /identity/v3/projects/51e1e1403bab4c50820547962b347992/users/ada68d81b7684081b5bf220290936d89/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:30.745171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0a517-9493-4203-9e6c-bc358069615c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:30.745497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0a517-9493-4203-9e6c-bc358069615c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:30.745733 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0a517-9493-4203-9e6c-bc358069615c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:30.793121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f8a0a517-9493-4203-9e6c-bc358069615c None None] MFA Rules not processed for user `ada68d81b7684081b5bf220290936d89`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:30.796161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f8a0a517-9493-4203-9e6c-bc358069615c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:30.959176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1388/2775] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:30.964369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ee744-cdeb-4d47-a68f-384bb9748749 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:31.064354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ee744-cdeb-4d47-a68f-384bb9748749 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:31.072798 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc44e0e-2f89-466f-8cfd-aaf1cf11a67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:31.073879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc44e0e-2f89-466f-8cfd-aaf1cf11a67d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:31.074383 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc44e0e-2f89-466f-8cfd-aaf1cf11a67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:31.074877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc44e0e-2f89-466f-8cfd-aaf1cf11a67d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:31.076257 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc44e0e-2f89-466f-8cfd-aaf1cf11a67d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:31.080124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc44e0e-2f89-466f-8cfd-aaf1cf11a67d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:31.129858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1388/2776] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:30 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:31.131278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0a517-9493-4203-9e6c-bc358069615c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:31.154597 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0a517-9493-4203-9e6c-bc358069615c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:31.243222 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:31.260748 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e66e0-ee8d-41a4-9d0a-b3509e3c8f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:31.261533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e66e0-ee8d-41a4-9d0a-b3509e3c8f4d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:31.262278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e66e0-ee8d-41a4-9d0a-b3509e3c8f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:31.262788 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e66e0-ee8d-41a4-9d0a-b3509e3c8f4d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:31.264050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e66e0-ee8d-41a4-9d0a-b3509e3c8f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:31.267849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e66e0-ee8d-41a4-9d0a-b3509e3c8f4d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:31.309108 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1389/2777] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:31 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:31.316207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc44e0e-2f89-466f-8cfd-aaf1cf11a67d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:31.329318 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc44e0e-2f89-466f-8cfd-aaf1cf11a67d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:31.339270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366660b-a105-43c1-850e-1c60ec89e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:31.339270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366660b-a105-43c1-850e-1c60ec89e879 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:31.339270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366660b-a105-43c1-850e-1c60ec89e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:31.339270 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366660b-a105-43c1-850e-1c60ec89e879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:31.339955 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366660b-a105-43c1-850e-1c60ec89e879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:31.341725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366660b-a105-43c1-850e-1c60ec89e879 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:31.354751 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1389/2778] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:31.359467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e66e0-ee8d-41a4-9d0a-b3509e3c8f4d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:31.371512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e66e0-ee8d-41a4-9d0a-b3509e3c8f4d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:31.377046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc3ef-abc1-4882-a472-c98e458e9015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:31.378036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc3ef-abc1-4882-a472-c98e458e9015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:31.378316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc3ef-abc1-4882-a472-c98e458e9015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:31.378575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc3ef-abc1-4882-a472-c98e458e9015 admin admin] PATH_INFO: `/v3/projects/c612c605af2a49b78044af038f87172e/users/2231f806e3f244758f27161891afc636/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:31.398270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fc3ef-abc1-4882-a472-c98e458e9015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c612c605af2a49b78044af038f87172e, user_id=2231f806e3f244758f27161891afc636, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:31.401122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fc3ef-abc1-4882-a472-c98e458e9015 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:31.419278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1390/2779] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:31 2022] PUT /identity/v3/projects/c612c605af2a49b78044af038f87172e/users/2231f806e3f244758f27161891afc636/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:31.425931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366660b-a105-43c1-850e-1c60ec89e879 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:31.440088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366660b-a105-43c1-850e-1c60ec89e879 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:31.532400 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:31.552500 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1221d-cc66-410c-bfee-6a4b8d99e263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:31.553551 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1221d-cc66-410c-bfee-6a4b8d99e263 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:31.554155 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1221d-cc66-410c-bfee-6a4b8d99e263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:31.554503 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1221d-cc66-410c-bfee-6a4b8d99e263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:31.555632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1221d-cc66-410c-bfee-6a4b8d99e263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:31.559050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1221d-cc66-410c-bfee-6a4b8d99e263 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:31.573631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1390/2780] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:54:31.579725 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc3ef-abc1-4882-a472-c98e458e9015 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:31.604423 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc3ef-abc1-4882-a472-c98e458e9015 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:31.626828 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:31.631096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:31.631802 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:31.632689 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] PATH_INFO: `/v3/projects/c612c605af2a49b78044af038f87172e/users/2231f806e3f244758f27161891afc636/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:31.646553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c612c605af2a49b78044af038f87172e, user_id=2231f806e3f244758f27161891afc636, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:31.652715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:31.667841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1391/2781] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:31 2022] PUT /identity/v3/projects/c612c605af2a49b78044af038f87172e/users/2231f806e3f244758f27161891afc636/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:31.688757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0142a0-ebe0-4f83-9203-b8f24a77a468 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:31.689668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0142a0-ebe0-4f83-9203-b8f24a77a468 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:31.689848 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0142a0-ebe0-4f83-9203-b8f24a77a468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:31.734594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-aa0142a0-ebe0-4f83-9203-b8f24a77a468 None None] MFA Rules not processed for user `2231f806e3f244758f27161891afc636`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:31.736301 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aa0142a0-ebe0-4f83-9203-b8f24a77a468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:31.819704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1391/2782] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:31.835798 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:31.837898 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:31.924411 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c753e5c-9c9a-4b51-9da9-d5efaf298157 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:31.928731 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e48c0-6463-4935-b783-c8eb80c08c09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:31.929447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e48c0-6463-4935-b783-c8eb80c08c09 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:31.929779 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e48c0-6463-4935-b783-c8eb80c08c09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:31.930093 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e48c0-6463-4935-b783-c8eb80c08c09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:31.941158 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e48c0-6463-4935-b783-c8eb80c08c09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:31.942707 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e48c0-6463-4935-b783-c8eb80c08c09 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:31.954299 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1392/2783] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:32.134617 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0142a0-ebe0-4f83-9203-b8f24a77a468 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:32.209803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0142a0-ebe0-4f83-9203-b8f24a77a468 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:32.215154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:32.215850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:32.216137 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:32.216405 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] PATH_INFO: `/v3/users/ada68d81b7684081b5bf220290936d89` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:32.217624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ada68d81b7684081b5bf220290936d89)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:32.219692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:32.279403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ada68d81b7684081b5bf220290936d89'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:32.280166 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ada68d81b7684081b5bf220290936d89'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:32.297555 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ada68d81b7684081b5bf220290936d89'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:32.310629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ada68d81b7684081b5bf220290936d89'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:32.313945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:32.331933 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ada68d81b7684081b5bf220290936d89'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:32.341127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1392/2784] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:32 2022] DELETE /identity/v3/users/ada68d81b7684081b5bf220290936d89 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:32.347365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880060-57a7-4804-aa3a-f78e13deaa62 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:32.348208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880060-57a7-4804-aa3a-f78e13deaa62 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:32.348208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880060-57a7-4804-aa3a-f78e13deaa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:32.377262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-85880060-57a7-4804-aa3a-f78e13deaa62 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:32.379157 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-85880060-57a7-4804-aa3a-f78e13deaa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:32.478642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1393/2785] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:32.495192 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:32.498247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:32.597506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bde10-2eec-4e61-a900-330b079f1025 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:32.603391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e1665-54ff-4811-92b7-63038e144e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:32.604341 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1665-54ff-4811-92b7-63038e144e68 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:32.604847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1665-54ff-4811-92b7-63038e144e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:32.605328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1665-54ff-4811-92b7-63038e144e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:32.622981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e1665-54ff-4811-92b7-63038e144e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:32.625823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e1665-54ff-4811-92b7-63038e144e68 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:32.645716 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1393/2786] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:32.992602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85880060-57a7-4804-aa3a-f78e13deaa62 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:32.994661 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-85880060-57a7-4804-aa3a-f78e13deaa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:33.013498 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85880060-57a7-4804-aa3a-f78e13deaa62 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:33.020658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:33.021324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:33.021607 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:33.021884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] PATH_INFO: `/v3/projects/51e1e1403bab4c50820547962b347992` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:33.024441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e1e1403bab4c50820547962b347992)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:33.026698 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:33.091768 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:33.097376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e1e1403bab4c50820547962b347992', 'request_id': 'req-f510813a-723f-4821-86ef-8a41b6ab650e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.106069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e1e1403bab4c50820547962b347992', 'request_id': 'req-f510813a-723f-4821-86ef-8a41b6ab650e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.117741 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e1e1403bab4c50820547962b347992', 'request_id': 'req-f510813a-723f-4821-86ef-8a41b6ab650e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.130649 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] The token cache is being invalidate because project 51e1e1403bab4c50820547962b347992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:33.130945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.131860 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.133526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1394/2787] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:32 2022] DELETE /identity/v3/projects/51e1e1403bab4c50820547962b347992 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:33.138370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e1665-54ff-4811-92b7-63038e144e68 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:33.150643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e1665-54ff-4811-92b7-63038e144e68 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:33.224128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:33.224804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:33.225151 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:33.225365 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] PATH_INFO: `/v3/users/2231f806e3f244758f27161891afc636` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:33.226866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2231f806e3f244758f27161891afc636)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:33.228889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:33.287990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2231f806e3f244758f27161891afc636'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.288757 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2231f806e3f244758f27161891afc636'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.304103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2231f806e3f244758f27161891afc636'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.312422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2231f806e3f244758f27161891afc636'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.321577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3433b355-d4d3-4745-a360-d272ee0a9d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2231f806e3f244758f27161891afc636'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:33.331883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1394/2788] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:33 2022] DELETE /identity/v3/users/2231f806e3f244758f27161891afc636 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:34.055899 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:34.061642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:34.083599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510813a-723f-4821-86ef-8a41b6ab650e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:34.162022 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:34.162885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.163258 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.163546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] PATH_INFO: `/v3/projects/c612c605af2a49b78044af038f87172e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.164821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c612c605af2a49b78044af038f87172e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:34.166817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:34.226422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c612c605af2a49b78044af038f87172e', 'request_id': 'req-47f091a8-7a97-42f3-b6a4-d092fe70265f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:34.235810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c612c605af2a49b78044af038f87172e', 'request_id': 'req-47f091a8-7a97-42f3-b6a4-d092fe70265f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:34.247212 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c612c605af2a49b78044af038f87172e', 'request_id': 'req-47f091a8-7a97-42f3-b6a4-d092fe70265f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:34.263585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] The token cache is being invalidate because project c612c605af2a49b78044af038f87172e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:34.264009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:34.264956 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f091a8-7a97-42f3-b6a4-d092fe70265f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:34.266592 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1395/2789] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:34 2022] DELETE /identity/v3/projects/c612c605af2a49b78044af038f87172e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:34.272351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54de647-8a7a-49c3-a612-9d41fb517f3c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.272967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54de647-8a7a-49c3-a612-9d41fb517f3c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.272967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54de647-8a7a-49c3-a612-9d41fb517f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.297064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c54de647-8a7a-49c3-a612-9d41fb517f3c None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:34.298293 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c54de647-8a7a-49c3-a612-9d41fb517f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:34.309713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:34.375432 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1395/2790] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:34.385253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e67483-c705-4efb-91a0-6be3561562d4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.385678 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e67483-c705-4efb-91a0-6be3561562d4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.386053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e67483-c705-4efb-91a0-6be3561562d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.409930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a9e67483-c705-4efb-91a0-6be3561562d4 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:34.411529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a9e67483-c705-4efb-91a0-6be3561562d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:34.422189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1396/2791] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:34.427708 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54de647-8a7a-49c3-a612-9d41fb517f3c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:34.438878 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54de647-8a7a-49c3-a612-9d41fb517f3c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:34.445828 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:34.446904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.447378 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.447750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.448606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:34.448999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:34.450779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:34.467723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1396/2792] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:34.478516 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e67483-c705-4efb-91a0-6be3561562d4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:34.489859 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e67483-c705-4efb-91a0-6be3561562d4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:34.496276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde2b9b-1670-4825-9ffa-4a184c4855ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:34.496912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde2b9b-1670-4825-9ffa-4a184c4855ca admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.500077 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde2b9b-1670-4825-9ffa-4a184c4855ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.500419 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde2b9b-1670-4825-9ffa-4a184c4855ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.501110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde2b9b-1670-4825-9ffa-4a184c4855ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:34.503856 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde2b9b-1670-4825-9ffa-4a184c4855ca admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:34.542161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1397/2793] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:34 2022] POST /identity/v3/projects => generated 361 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:34.546261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:34.558035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa1ee-49c9-4948-a448-805c4ced9d8d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:34.628679 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037fdd0-c97d-4236-bca0-6b3081724c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:34.629511 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037fdd0-c97d-4236-bca0-6b3081724c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.629788 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037fdd0-c97d-4236-bca0-6b3081724c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.630071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037fdd0-c97d-4236-bca0-6b3081724c3d admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.630767 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037fdd0-c97d-4236-bca0-6b3081724c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:34.632779 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037fdd0-c97d-4236-bca0-6b3081724c3d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:34.656876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1397/2794] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:34 2022] POST /identity/v3/users => generated 281 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:34.662610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde2b9b-1670-4825-9ffa-4a184c4855ca admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:34.675106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde2b9b-1670-4825-9ffa-4a184c4855ca admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:34.681428 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfb20d-af25-4476-9268-e1b5b1ba8e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:34.682116 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfb20d-af25-4476-9268-e1b5b1ba8e85 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.682381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfb20d-af25-4476-9268-e1b5b1ba8e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.682654 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfb20d-af25-4476-9268-e1b5b1ba8e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.683372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfb20d-af25-4476-9268-e1b5b1ba8e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:34.685183 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfb20d-af25-4476-9268-e1b5b1ba8e85 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:34.696952 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1398/2795] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:34.701350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037fdd0-c97d-4236-bca0-6b3081724c3d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:34.713678 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037fdd0-c97d-4236-bca0-6b3081724c3d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:34.719147 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19e2e9-9e1e-4969-ae8d-15cf40ab1e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:34.719819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19e2e9-9e1e-4969-ae8d-15cf40ab1e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.720096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19e2e9-9e1e-4969-ae8d-15cf40ab1e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.720355 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19e2e9-9e1e-4969-ae8d-15cf40ab1e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.721173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19e2e9-9e1e-4969-ae8d-15cf40ab1e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:34.724018 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19e2e9-9e1e-4969-ae8d-15cf40ab1e4b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:34.738446 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1398/2796] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:34.745716 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfb20d-af25-4476-9268-e1b5b1ba8e85 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:34.763994 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfb20d-af25-4476-9268-e1b5b1ba8e85 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:34.779628 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:34.780766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.781243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.781711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] PATH_INFO: `/v3/projects/4ab9993e50f24996a35d09b41913dbd8/users/b450013bc81947d89d44e8fcd7548f69/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.810620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab9993e50f24996a35d09b41913dbd8, user_id=b450013bc81947d89d44e8fcd7548f69, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:34.815544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:34.832053 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:34.838904 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1399/2797] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:34 2022] PUT /identity/v3/projects/4ab9993e50f24996a35d09b41913dbd8/users/b450013bc81947d89d44e8fcd7548f69/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:34.852332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d033b8-6807-4c0c-bea0-1c30a4fae727 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:34.852332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d033b8-6807-4c0c-bea0-1c30a4fae727 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:34.852766 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d033b8-6807-4c0c-bea0-1c30a4fae727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:34.904144 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c7d033b8-6807-4c0c-bea0-1c30a4fae727 None None] MFA Rules not processed for user `b450013bc81947d89d44e8fcd7548f69`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:34.905969 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c7d033b8-6807-4c0c-bea0-1c30a4fae727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:35.091021 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1399/2798] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:35.158699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:35.161758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:35.276481 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625e732-cae6-4fa3-838b-6e0432e41a51 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:35.282414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf760de-f26f-4f8f-9f26-280ae3d69935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:35.283462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf760de-f26f-4f8f-9f26-280ae3d69935 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:35.283979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf760de-f26f-4f8f-9f26-280ae3d69935 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:35.284450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf760de-f26f-4f8f-9f26-280ae3d69935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:35.299771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf760de-f26f-4f8f-9f26-280ae3d69935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:35.302119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf760de-f26f-4f8f-9f26-280ae3d69935 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:35.319706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1400/2799] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:35.342044 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d033b8-6807-4c0c-bea0-1c30a4fae727 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:35.360483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:35.465100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d033b8-6807-4c0c-bea0-1c30a4fae727 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:35.470670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:35.471831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:35.472533 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:35.473926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] PATH_INFO: `/v3/users/b450013bc81947d89d44e8fcd7548f69` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:35.475313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b450013bc81947d89d44e8fcd7548f69)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:35.478332 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:35.604938 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b450013bc81947d89d44e8fcd7548f69'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:35.606319 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b450013bc81947d89d44e8fcd7548f69'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:35.645057 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b450013bc81947d89d44e8fcd7548f69'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:35.661417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b450013bc81947d89d44e8fcd7548f69'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:35.674932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b450013bc81947d89d44e8fcd7548f69'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:35.683610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1400/2800] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:35 2022] DELETE /identity/v3/users/b450013bc81947d89d44e8fcd7548f69 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:35.690041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d274da-130c-4edb-9ac8-48b58b910cb3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:35.690520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d274da-130c-4edb-9ac8-48b58b910cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:35.691202 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d274da-130c-4edb-9ac8-48b58b910cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:35.716236 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e1d274da-130c-4edb-9ac8-48b58b910cb3 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:35.718664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e1d274da-130c-4edb-9ac8-48b58b910cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:35.830835 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1401/2801] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:35.841794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:35.844002 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:35.925631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc05eb4-4061-4bda-9de3-8eb82c2f7c9f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:35.929699 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834aeaf-2336-4d9d-b1b7-f24f556a3807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:35.930505 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834aeaf-2336-4d9d-b1b7-f24f556a3807 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:35.930960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834aeaf-2336-4d9d-b1b7-f24f556a3807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:35.931472 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834aeaf-2336-4d9d-b1b7-f24f556a3807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:35.942898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834aeaf-2336-4d9d-b1b7-f24f556a3807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:35.944649 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834aeaf-2336-4d9d-b1b7-f24f556a3807 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:35.956870 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1401/2802] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:36.451039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d274da-130c-4edb-9ac8-48b58b910cb3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:36.455154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e1d274da-130c-4edb-9ac8-48b58b910cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:36.481887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d274da-130c-4edb-9ac8-48b58b910cb3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:36.487422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:36.488469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:36.488816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:36.489154 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] PATH_INFO: `/v3/projects/4ab9993e50f24996a35d09b41913dbd8` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:36.490578 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab9993e50f24996a35d09b41913dbd8)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:36.493126 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:36.554976 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab9993e50f24996a35d09b41913dbd8', 'request_id': 'req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:36.562664 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab9993e50f24996a35d09b41913dbd8', 'request_id': 'req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:36.574370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab9993e50f24996a35d09b41913dbd8', 'request_id': 'req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:36.589693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] The token cache is being invalidate because project 4ab9993e50f24996a35d09b41913dbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:36.589978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:36.590721 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4fc3eb8-01e2-4a98-ba97-d534290b60d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:36.592320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1402/2803] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:36 2022] DELETE /identity/v3/projects/4ab9993e50f24996a35d09b41913dbd8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:36.598926 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c4c5-1c5a-4ded-ab6d-f36cf38e9b70 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:36.599773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c4c5-1c5a-4ded-ab6d-f36cf38e9b70 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:36.599948 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c4c5-1c5a-4ded-ab6d-f36cf38e9b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:36.625049 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e424c4c5-1c5a-4ded-ab6d-f36cf38e9b70 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:36.627196 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e424c4c5-1c5a-4ded-ab6d-f36cf38e9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:36.668572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:36.740184 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1402/2804] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:36.750782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2d0f4-f51e-4aa7-b94f-a21e2dcf822e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:36.751541 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2d0f4-f51e-4aa7-b94f-a21e2dcf822e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:36.752177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2d0f4-f51e-4aa7-b94f-a21e2dcf822e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:36.792147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-16b2d0f4-f51e-4aa7-b94f-a21e2dcf822e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:36.794276 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-16b2d0f4-f51e-4aa7-b94f-a21e2dcf822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:36.809023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1403/2805] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:36.814091 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c4c5-1c5a-4ded-ab6d-f36cf38e9b70 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:36.834202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c4c5-1c5a-4ded-ab6d-f36cf38e9b70 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:36.840953 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:36.841539 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:36.841899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:36.842359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:36.843173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:36.843560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:36.845554 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:36.865313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1403/2806] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:36.871424 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2d0f4-f51e-4aa7-b94f-a21e2dcf822e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:36.886450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2d0f4-f51e-4aa7-b94f-a21e2dcf822e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:36.900850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef54b-a6ad-4143-ade1-7b7e6faa480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:36.902027 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef54b-a6ad-4143-ade1-7b7e6faa480e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:36.902462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef54b-a6ad-4143-ade1-7b7e6faa480e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:36.903169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef54b-a6ad-4143-ade1-7b7e6faa480e admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:36.904152 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ef54b-a6ad-4143-ade1-7b7e6faa480e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:36.906961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ef54b-a6ad-4143-ade1-7b7e6faa480e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:36.960469 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:36.973743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1404/2807] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:36 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:36.979907 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:36.992713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c104d0-3253-45ba-b772-90363f4f5465 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:37.066717 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019e55e-0ec3-4a5d-bfc8-e0c7e587a7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:37.067490 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019e55e-0ec3-4a5d-bfc8-e0c7e587a7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:37.067812 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019e55e-0ec3-4a5d-bfc8-e0c7e587a7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:37.068122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019e55e-0ec3-4a5d-bfc8-e0c7e587a7f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:37.068819 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019e55e-0ec3-4a5d-bfc8-e0c7e587a7f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:37.070942 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019e55e-0ec3-4a5d-bfc8-e0c7e587a7f2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:37.099967 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1404/2808] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:36 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:37.106630 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef54b-a6ad-4143-ade1-7b7e6faa480e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:37.118319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef54b-a6ad-4143-ade1-7b7e6faa480e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:37.124137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fd990-6ef4-4b1a-929b-19669575211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:37.124575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fd990-6ef4-4b1a-929b-19669575211f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:37.124977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fd990-6ef4-4b1a-929b-19669575211f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:37.125245 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fd990-6ef4-4b1a-929b-19669575211f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:37.125940 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fd990-6ef4-4b1a-929b-19669575211f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:37.128082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fd990-6ef4-4b1a-929b-19669575211f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:37.141494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1405/2809] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:54:37.146362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019e55e-0ec3-4a5d-bfc8-e0c7e587a7f2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:37.158491 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019e55e-0ec3-4a5d-bfc8-e0c7e587a7f2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:37.165195 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6349ed9-1b21-4175-af5d-eceb8f73febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:37.165846 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6349ed9-1b21-4175-af5d-eceb8f73febc admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:37.166125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6349ed9-1b21-4175-af5d-eceb8f73febc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:37.166416 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6349ed9-1b21-4175-af5d-eceb8f73febc admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:37.167081 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6349ed9-1b21-4175-af5d-eceb8f73febc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:37.168916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6349ed9-1b21-4175-af5d-eceb8f73febc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:37.184279 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1405/2810] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:37.190266 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fd990-6ef4-4b1a-929b-19669575211f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:37.202282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fd990-6ef4-4b1a-929b-19669575211f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:37.208658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:37.209510 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:37.209968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:37.210465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] PATH_INFO: `/v3/projects/e48385693faf49e6a46415b6cd7205d8/users/e401a2812b8040ad8fc0b1553f0e1f65/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:37.233973 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48385693faf49e6a46415b6cd7205d8, user_id=e401a2812b8040ad8fc0b1553f0e1f65, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:37.237823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:37.286569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1406/2811] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:37 2022] PUT /identity/v3/projects/e48385693faf49e6a46415b6cd7205d8/users/e401a2812b8040ad8fc0b1553f0e1f65/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:54:37.304791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8c5f-302e-4907-9914-b23b672ce16a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:37.305204 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8c5f-302e-4907-9914-b23b672ce16a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:37.305654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8c5f-302e-4907-9914-b23b672ce16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:37.349577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-991f8c5f-302e-4907-9914-b23b672ce16a None None] MFA Rules not processed for user `e401a2812b8040ad8fc0b1553f0e1f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:37.351674 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-991f8c5f-302e-4907-9914-b23b672ce16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:37.452068 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1406/2812] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:37.469717 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:37.472642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:37.563680 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d0e1-94c0-4eae-9006-8087d74a7b1f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:37.568577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086458e9-74be-46fb-8904-44987ee40f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:37.569438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086458e9-74be-46fb-8904-44987ee40f31 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:37.569815 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086458e9-74be-46fb-8904-44987ee40f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:37.570253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086458e9-74be-46fb-8904-44987ee40f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:37.581513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086458e9-74be-46fb-8904-44987ee40f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:37.583017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086458e9-74be-46fb-8904-44987ee40f31 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:37.593629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1407/2813] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:40.442398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f8c5f-302e-4907-9914-b23b672ce16a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:40.444949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-991f8c5f-302e-4907-9914-b23b672ce16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:40.503430 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:40.578640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f8c5f-302e-4907-9914-b23b672ce16a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:40.584981 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19811f-2e5b-4d93-bf12-49e5cf4128b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:40.585670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19811f-2e5b-4d93-bf12-49e5cf4128b1 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:40.585987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19811f-2e5b-4d93-bf12-49e5cf4128b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:40.586312 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19811f-2e5b-4d93-bf12-49e5cf4128b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:40.588453 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ba19811f-2e5b-4d93-bf12-49e5cf4128b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:40.602774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19811f-2e5b-4d93-bf12-49e5cf4128b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:40.604581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19811f-2e5b-4d93-bf12-49e5cf4128b1 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:40.619283 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1407/2814] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:43.232422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086458e9-74be-46fb-8904-44987ee40f31 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:43.316440 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086458e9-74be-46fb-8904-44987ee40f31 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:43.322206 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:43.322862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:43.323141 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:43.323496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] PATH_INFO: `/v3/users/e401a2812b8040ad8fc0b1553f0e1f65` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:43.324797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e401a2812b8040ad8fc0b1553f0e1f65)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:43.327255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:43.392968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e401a2812b8040ad8fc0b1553f0e1f65'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:43.394269 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e401a2812b8040ad8fc0b1553f0e1f65'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:43.406384 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e401a2812b8040ad8fc0b1553f0e1f65'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:43.407187 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e401a2812b8040ad8fc0b1553f0e1f65'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:43.426582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:43.430071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e401a2812b8040ad8fc0b1553f0e1f65'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:43.450219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1408/2815] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:43 2022] DELETE /identity/v3/users/e401a2812b8040ad8fc0b1553f0e1f65 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:43.455801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da017e1f-bd35-4a20-9c6f-abf4b641ed33 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:43.456718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da017e1f-bd35-4a20-9c6f-abf4b641ed33 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:43.456993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da017e1f-bd35-4a20-9c6f-abf4b641ed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:43.480336 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-da017e1f-bd35-4a20-9c6f-abf4b641ed33 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:43.481622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-da017e1f-bd35-4a20-9c6f-abf4b641ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:43.600573 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1408/2816] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:43.615810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:43.618284 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:43.720789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdec5e8-3f3a-4edc-bc0b-2656d28ec46c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:43.724833 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489a48a-ee79-4b14-85a4-61119ec47650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:43.725459 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a48a-ee79-4b14-85a4-61119ec47650 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:43.725732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a48a-ee79-4b14-85a4-61119ec47650 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:43.725995 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a48a-ee79-4b14-85a4-61119ec47650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:43.736948 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489a48a-ee79-4b14-85a4-61119ec47650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:43.738595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489a48a-ee79-4b14-85a4-61119ec47650 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:43.750134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1409/2817] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:54:44.204597 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da017e1f-bd35-4a20-9c6f-abf4b641ed33 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:44.211790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-da017e1f-bd35-4a20-9c6f-abf4b641ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:44.236725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da017e1f-bd35-4a20-9c6f-abf4b641ed33 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:44.248586 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:44.249397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.249801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.250259 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] PATH_INFO: `/v3/projects/e48385693faf49e6a46415b6cd7205d8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.256979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e48385693faf49e6a46415b6cd7205d8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:44.265130 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:44.343864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e48385693faf49e6a46415b6cd7205d8', 'request_id': 'req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:44.357803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e48385693faf49e6a46415b6cd7205d8', 'request_id': 'req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:44.372827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e48385693faf49e6a46415b6cd7205d8', 'request_id': 'req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:44.393633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] The token cache is being invalidate because project e48385693faf49e6a46415b6cd7205d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:44.394247 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:44.395529 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fa1a032d-7fc1-46d0-aa2e-86a5f1de7337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:44.398481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1409/2818] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:44 2022] DELETE /identity/v3/projects/e48385693faf49e6a46415b6cd7205d8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:44.406438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2df8b-e861-4d4f-9da6-7f49824f8c3f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.407007 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2df8b-e861-4d4f-9da6-7f49824f8c3f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.407443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2df8b-e861-4d4f-9da6-7f49824f8c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.442118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-95c2df8b-e861-4d4f-9da6-7f49824f8c3f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:44.443704 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-95c2df8b-e861-4d4f-9da6-7f49824f8c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:44.523941 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1410/2819] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:44.539457 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6467fe-ca6c-4061-9892-50e107746499 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.540290 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6467fe-ca6c-4061-9892-50e107746499 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.540473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6467fe-ca6c-4061-9892-50e107746499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.572827 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ad6467fe-ca6c-4061-9892-50e107746499 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:44.574150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad6467fe-ca6c-4061-9892-50e107746499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:44.587962 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1410/2820] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:44.592789 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2df8b-e861-4d4f-9da6-7f49824f8c3f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:44.610278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2df8b-e861-4d4f-9da6-7f49824f8c3f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:44.618811 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:44.619905 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.620313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.620776 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.622003 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:44.622566 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:44.626827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:44.643939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1411/2821] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:54:44.649236 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6467fe-ca6c-4061-9892-50e107746499 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:44.660000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6467fe-ca6c-4061-9892-50e107746499 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:44.665001 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c837669-7f7a-4a2c-8ff4-00edd576a457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:44.665640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c837669-7f7a-4a2c-8ff4-00edd576a457 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.665914 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c837669-7f7a-4a2c-8ff4-00edd576a457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.666179 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c837669-7f7a-4a2c-8ff4-00edd576a457 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.666864 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c837669-7f7a-4a2c-8ff4-00edd576a457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:44.668899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c837669-7f7a-4a2c-8ff4-00edd576a457 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:44.703906 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1411/2822] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:44 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:44.708778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:44.719844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd19d0d-678f-4075-b8cd-14d743b330b8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:44.791969 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a6b9b-1a5e-4451-b0e7-969c7ae6dbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:44.792536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a6b9b-1a5e-4451-b0e7-969c7ae6dbdd admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.793122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a6b9b-1a5e-4451-b0e7-969c7ae6dbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.793395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a6b9b-1a5e-4451-b0e7-969c7ae6dbdd admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.794091 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a6b9b-1a5e-4451-b0e7-969c7ae6dbdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:44.796171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a6b9b-1a5e-4451-b0e7-969c7ae6dbdd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:44.824451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1412/2823] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:44 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:44.829289 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c837669-7f7a-4a2c-8ff4-00edd576a457 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:44.842328 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c837669-7f7a-4a2c-8ff4-00edd576a457 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:44.849011 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03359d83-6c93-42f1-a8ea-feb634349b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:44.849644 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03359d83-6c93-42f1-a8ea-feb634349b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.849921 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03359d83-6c93-42f1-a8ea-feb634349b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.850187 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03359d83-6c93-42f1-a8ea-feb634349b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.850845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03359d83-6c93-42f1-a8ea-feb634349b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:44.852588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03359d83-6c93-42f1-a8ea-feb634349b51 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:44.854327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:44.868354 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1412/2824] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:44.874502 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a6b9b-1a5e-4451-b0e7-969c7ae6dbdd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:44.885506 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:44.893857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a6b9b-1a5e-4451-b0e7-969c7ae6dbdd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:44.904932 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450ad5d-6875-4038-be1c-f0661ec895a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:44.906520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ad5d-6875-4038-be1c-f0661ec895a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.907080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ad5d-6875-4038-be1c-f0661ec895a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.907774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ad5d-6875-4038-be1c-f0661ec895a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.909310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450ad5d-6875-4038-be1c-f0661ec895a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:44.912864 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450ad5d-6875-4038-be1c-f0661ec895a0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:44.930961 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1413/2825] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:44.935604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03359d83-6c93-42f1-a8ea-feb634349b51 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:44.950009 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03359d83-6c93-42f1-a8ea-feb634349b51 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:44.957342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:44.963792 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:44.964261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:44.964753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] PATH_INFO: `/v3/projects/e025dc367b3a4349b1f708b7b3966d43/users/48ae2b062eb644059edfa65d61f60f78/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:44.997495 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e025dc367b3a4349b1f708b7b3966d43, user_id=48ae2b062eb644059edfa65d61f60f78, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:45.002794 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:45.026038 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1413/2826] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:44 2022] PUT /identity/v3/projects/e025dc367b3a4349b1f708b7b3966d43/users/48ae2b062eb644059edfa65d61f60f78/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:45.032029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f964ff-6334-42a1-9d13-1cb5c88bba3e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:45.032374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f964ff-6334-42a1-9d13-1cb5c88bba3e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:45.032700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f964ff-6334-42a1-9d13-1cb5c88bba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:45.057690 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c6f964ff-6334-42a1-9d13-1cb5c88bba3e None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:45.058884 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c6f964ff-6334-42a1-9d13-1cb5c88bba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:45.161102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1414/2827] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:45.174124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:45.177113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:45.290129 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fb2f0-315d-41d2-b5f9-770e503c7055 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:45.295839 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d0b8f-cc49-4cf0-8f94-9a3c890b3fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:45.296588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0b8f-cc49-4cf0-8f94-9a3c890b3fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:45.296972 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0b8f-cc49-4cf0-8f94-9a3c890b3fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:45.297424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0b8f-cc49-4cf0-8f94-9a3c890b3fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:45.319351 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d0b8f-cc49-4cf0-8f94-9a3c890b3fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:45.322084 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d0b8f-cc49-4cf0-8f94-9a3c890b3fb5 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:45.345097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1414/2828] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:54:45.603372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f964ff-6334-42a1-9d13-1cb5c88bba3e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:45.605066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c6f964ff-6334-42a1-9d13-1cb5c88bba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:45.628396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f964ff-6334-42a1-9d13-1cb5c88bba3e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:45.640029 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:45.641076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:45.641551 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:45.642014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] PATH_INFO: `/v3/users/c8a1d291e6be4785b9178e7db6c0056c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:45.644480 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a1d291e6be4785b9178e7db6c0056c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:45.648112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:45.734586 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a1d291e6be4785b9178e7db6c0056c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:45.735615 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a1d291e6be4785b9178e7db6c0056c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:45.751450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a1d291e6be4785b9178e7db6c0056c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:45.755208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a1d291e6be4785b9178e7db6c0056c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:45.778401 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a1d291e6be4785b9178e7db6c0056c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:45.809965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1415/2829] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:54:45 2022] DELETE /identity/v3/users/c8a1d291e6be4785b9178e7db6c0056c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:45.818584 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62642-5983-4f4e-8b80-9d9dfd85131d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:45.820010 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62642-5983-4f4e-8b80-9d9dfd85131d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:45.821041 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62642-5983-4f4e-8b80-9d9dfd85131d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:45.882267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3ef62642-5983-4f4e-8b80-9d9dfd85131d None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:45.890919 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3ef62642-5983-4f4e-8b80-9d9dfd85131d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:46.110745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1415/2830] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:46.126438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:46.129827 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:46.268295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b1f4b-3c96-432a-9d2f-5041d255f57a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:46.275467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:46.276727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:46.277280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:46.277918 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:46.298373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:46.301946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:46.327279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1416/2831] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:46.863823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24878-440b-4d9e-b8db-d4f6f1fbe3d5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:46.874621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24878-440b-4d9e-b8db-d4f6f1fbe3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:46.881934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24878-440b-4d9e-b8db-d4f6f1fbe3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:46.912885 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:46.916435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:46.972700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9822-7b55-4a23-9872-954ef9d76c34 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:46.973820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cdc24878-440b-4d9e-b8db-d4f6f1fbe3d5 None None] MFA Rules not processed for user `48ae2b062eb644059edfa65d61f60f78`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:46.977113 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cdc24878-440b-4d9e-b8db-d4f6f1fbe3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:46.987395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:46.988527 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:46.989023 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:46.989517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] PATH_INFO: `/v3/projects/513f5ad5cf9c497d987e48d92bafa0ad` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:46.998695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=513f5ad5cf9c497d987e48d92bafa0ad)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:47.014112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:47.022514 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:47.115315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1416/2832] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:47.132317 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24878-440b-4d9e-b8db-d4f6f1fbe3d5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:47.135096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cdc24878-440b-4d9e-b8db-d4f6f1fbe3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:47.137774 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513f5ad5cf9c497d987e48d92bafa0ad', 'request_id': 'req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:47.147308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513f5ad5cf9c497d987e48d92bafa0ad', 'request_id': 'req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:47.152823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:47.162381 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513f5ad5cf9c497d987e48d92bafa0ad', 'request_id': 'req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:47.188782 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] The token cache is being invalidate because project 513f5ad5cf9c497d987e48d92bafa0ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:54:47.189137 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:47.189911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e57ac6d5-1c76-4e6a-aa51-7d15f6bb8444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:54:47.191764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1417/2833] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:54:46 2022] DELETE /identity/v3/projects/513f5ad5cf9c497d987e48d92bafa0ad => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:47.197536 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de8bc5-2d2e-407a-a300-a5c352b06254 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.197796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de8bc5-2d2e-407a-a300-a5c352b06254 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.198079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de8bc5-2d2e-407a-a300-a5c352b06254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.224478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-14de8bc5-2d2e-407a-a300-a5c352b06254 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:47.226575 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-14de8bc5-2d2e-407a-a300-a5c352b06254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:47.262112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24878-440b-4d9e-b8db-d4f6f1fbe3d5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:47.313070 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1418/2834] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:47.323002 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f034ca-f7a7-421e-8b69-8c7607b81ce2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.323483 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f034ca-f7a7-421e-8b69-8c7607b81ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.323781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f034ca-f7a7-421e-8b69-8c7607b81ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.350189 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d0f034ca-f7a7-421e-8b69-8c7607b81ce2 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:47.351787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d0f034ca-f7a7-421e-8b69-8c7607b81ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:47.360641 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:47.362075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.362727 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.363297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.363596 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1419/2835] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:47.366485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f034ca-f7a7-421e-8b69-8c7607b81ce2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:47.368861 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:47.377587 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f034ca-f7a7-421e-8b69-8c7607b81ce2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:47.383341 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:47.383917 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.384193 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.384462 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.385125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:54:47.385431 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:47.387098 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:47.408342 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1420/2836] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:54:47.414346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:47.430549 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d72a-aca8-4aa1-bc88-49c79386f734 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:47.436781 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c1d3b-55b7-4c0b-8151-a2dcd9c5da49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:47.437407 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1d3b-55b7-4c0b-8151-a2dcd9c5da49 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.437686 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1d3b-55b7-4c0b-8151-a2dcd9c5da49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.437968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1d3b-55b7-4c0b-8151-a2dcd9c5da49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.438632 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c1d3b-55b7-4c0b-8151-a2dcd9c5da49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:47.441347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c1d3b-55b7-4c0b-8151-a2dcd9c5da49 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:47.452561 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:47.454039 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:47.464501 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1417/2837] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:47.494204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1421/2838] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:47 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:54:47.495116 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:47.514836 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8812e9-5ce6-4e1f-acd9-98561e380716 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:47.603711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a88615-79f2-43e7-88ee-06865ce71fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:47.604356 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a88615-79f2-43e7-88ee-06865ce71fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.604615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a88615-79f2-43e7-88ee-06865ce71fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.604883 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a88615-79f2-43e7-88ee-06865ce71fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.605583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a88615-79f2-43e7-88ee-06865ce71fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:47.607715 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a88615-79f2-43e7-88ee-06865ce71fc6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:47.654390 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1418/2839] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:47 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:47.660350 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c1d3b-55b7-4c0b-8151-a2dcd9c5da49 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:47.681959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c1d3b-55b7-4c0b-8151-a2dcd9c5da49 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:47.691874 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96c4c4-9af3-44c6-8e52-d57ca7eb0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:47.692610 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96c4c4-9af3-44c6-8e52-d57ca7eb0f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.692861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96c4c4-9af3-44c6-8e52-d57ca7eb0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.693101 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96c4c4-9af3-44c6-8e52-d57ca7eb0f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.693797 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96c4c4-9af3-44c6-8e52-d57ca7eb0f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:47.695739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96c4c4-9af3-44c6-8e52-d57ca7eb0f32 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:47.709345 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1422/2840] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:54:47.715632 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a88615-79f2-43e7-88ee-06865ce71fc6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:47.728522 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:47.737990 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a88615-79f2-43e7-88ee-06865ce71fc6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:47.749676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b9efe-7396-4700-9a9a-41f4619cb3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:47.751711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b9efe-7396-4700-9a9a-41f4619cb3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.752323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b9efe-7396-4700-9a9a-41f4619cb3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.753750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b9efe-7396-4700-9a9a-41f4619cb3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.755222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b9efe-7396-4700-9a9a-41f4619cb3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:47.758482 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b9efe-7396-4700-9a9a-41f4619cb3b3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:47.779150 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1419/2841] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:47.785000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96c4c4-9af3-44c6-8e52-d57ca7eb0f32 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:47.796316 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96c4c4-9af3-44c6-8e52-d57ca7eb0f32 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:47.801819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:47.802534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.802806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.803693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] PATH_INFO: `/v3/projects/b5e2d3e145e04936bf60fe93b5011cc2/users/171b9c24597545a1a5c8388849ef5819/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.835314 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e2d3e145e04936bf60fe93b5011cc2, user_id=171b9c24597545a1a5c8388849ef5819, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:47.842272 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:47.870310 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1423/2842] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:47 2022] PUT /identity/v3/projects/b5e2d3e145e04936bf60fe93b5011cc2/users/171b9c24597545a1a5c8388849ef5819/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:47.878905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8704e99-e23a-41b3-9e05-cbf40348f488 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:47.879488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8704e99-e23a-41b3-9e05-cbf40348f488 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:47.879742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8704e99-e23a-41b3-9e05-cbf40348f488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:47.910916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c8704e99-e23a-41b3-9e05-cbf40348f488 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:47.912109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c8704e99-e23a-41b3-9e05-cbf40348f488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:48.037251 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1420/2843] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:54:48.044221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:48.046843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:48.074507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:48.129849 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeee3e-9214-4b3f-bed5-f568a89daed8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:48.133785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ddecd-1bd2-499e-bce0-5ac83d1752dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:48.134400 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ddecd-1bd2-499e-bce0-5ac83d1752dc service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:48.134677 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ddecd-1bd2-499e-bce0-5ac83d1752dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:48.134944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ddecd-1bd2-499e-bce0-5ac83d1752dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:48.146724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ddecd-1bd2-499e-bce0-5ac83d1752dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:48.148326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ddecd-1bd2-499e-bce0-5ac83d1752dc service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:48.160504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1424/2844] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:49.156713 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda23f8-4246-4810-bea5-74edbb2267c0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:49.157607 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda23f8-4246-4810-bea5-74edbb2267c0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:49.157743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda23f8-4246-4810-bea5-74edbb2267c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:49.196101 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0cda23f8-4246-4810-bea5-74edbb2267c0 None None] MFA Rules not processed for user `48ae2b062eb644059edfa65d61f60f78`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:49.198273 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0cda23f8-4246-4810-bea5-74edbb2267c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:49.321212 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1421/2845] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:50.168848 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ddecd-1bd2-499e-bce0-5ac83d1752dc service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:50.183186 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ddecd-1bd2-499e-bce0-5ac83d1752dc service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:50.190796 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:50.192946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:50.192946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:50.192946 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:50.195150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:50.215296 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:50.218376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:50.236497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1425/2846] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:54:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:54:55.981854 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae7e1a-b6bd-4d4a-975b-09ccee3d80ec None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:55.982434 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae7e1a-b6bd-4d4a-975b-09ccee3d80ec None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:55.982773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae7e1a-b6bd-4d4a-975b-09ccee3d80ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:56.054380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c6ae7e1a-b6bd-4d4a-975b-09ccee3d80ec None None] MFA Rules not processed for user `171b9c24597545a1a5c8388849ef5819`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:54:56.056437 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c6ae7e1a-b6bd-4d4a-975b-09ccee3d80ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:54:56.376860 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1422/2847] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:54:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:54:56.379388 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:56.399436 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003d4a5-81fa-455c-8404-1e70bd163767 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:56.405795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd9f21-34f8-40ab-8e79-f4c6185f3e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:56.406862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd9f21-34f8-40ab-8e79-f4c6185f3e9e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:56.407422 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd9f21-34f8-40ab-8e79-f4c6185f3e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:56.407912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd9f21-34f8-40ab-8e79-f4c6185f3e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:56.409036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd9f21-34f8-40ab-8e79-f4c6185f3e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:56.411921 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd9f21-34f8-40ab-8e79-f4c6185f3e9e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:56.462881 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1426/2848] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:54:56 2022] POST /identity/v3/projects => generated 379 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:56.464831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae7e1a-b6bd-4d4a-975b-09ccee3d80ec None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:56.496458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae7e1a-b6bd-4d4a-975b-09ccee3d80ec None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:56.610483 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:54:56.695368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d8e8-8f81-4b76-8c96-55e0f309a38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:56.696115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d8e8-8f81-4b76-8c96-55e0f309a38e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:56.696465 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d8e8-8f81-4b76-8c96-55e0f309a38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:56.696739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d8e8-8f81-4b76-8c96-55e0f309a38e admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:56.697427 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66d8e8-8f81-4b76-8c96-55e0f309a38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:56.699620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66d8e8-8f81-4b76-8c96-55e0f309a38e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:56.734497 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1423/2849] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:54:56 2022] POST /identity/v3/users => generated 296 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:54:56.736801 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd9f21-34f8-40ab-8e79-f4c6185f3e9e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:56.761408 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd9f21-34f8-40ab-8e79-f4c6185f3e9e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:56.769393 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996dfdd4-1c19-46a3-ad15-da09cd3f1ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:56.770576 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dfdd4-1c19-46a3-ad15-da09cd3f1ece admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:56.771308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dfdd4-1c19-46a3-ad15-da09cd3f1ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:56.771986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dfdd4-1c19-46a3-ad15-da09cd3f1ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:56.773097 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996dfdd4-1c19-46a3-ad15-da09cd3f1ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:56.775504 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996dfdd4-1c19-46a3-ad15-da09cd3f1ece admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:56.791775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1427/2850] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:56.801121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d8e8-8f81-4b76-8c96-55e0f309a38e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:56.818660 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d8e8-8f81-4b76-8c96-55e0f309a38e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:56.828633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704613a-3dc8-4cbe-8e54-8b6ffc9d575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:56.828965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704613a-3dc8-4cbe-8e54-8b6ffc9d575d admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:56.829135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704613a-3dc8-4cbe-8e54-8b6ffc9d575d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:56.829479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704613a-3dc8-4cbe-8e54-8b6ffc9d575d admin admin] PATH_INFO: `/v3/projects/63297fdcd97b4e2181d3d2fcde70c516/users/7c36cbb15c5c49d0a1ea96c6cc0b9ab8/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:56.879833 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704613a-3dc8-4cbe-8e54-8b6ffc9d575d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63297fdcd97b4e2181d3d2fcde70c516, user_id=7c36cbb15c5c49d0a1ea96c6cc0b9ab8, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:56.882682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704613a-3dc8-4cbe-8e54-8b6ffc9d575d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:56.904888 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1424/2851] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:56 2022] PUT /identity/v3/projects/63297fdcd97b4e2181d3d2fcde70c516/users/7c36cbb15c5c49d0a1ea96c6cc0b9ab8/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:54:56.910699 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996dfdd4-1c19-46a3-ad15-da09cd3f1ece admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:56.923847 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996dfdd4-1c19-46a3-ad15-da09cd3f1ece admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:57.054288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991fd0f-a377-40cc-b9b5-57a452586c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:57.054288 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991fd0f-a377-40cc-b9b5-57a452586c16 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:57.055213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991fd0f-a377-40cc-b9b5-57a452586c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:57.055457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991fd0f-a377-40cc-b9b5-57a452586c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:57.056902 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991fd0f-a377-40cc-b9b5-57a452586c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:57.059215 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991fd0f-a377-40cc-b9b5-57a452586c16 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:57.078430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1428/2852] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:54:57.083843 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704613a-3dc8-4cbe-8e54-8b6ffc9d575d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:54:57.111124 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704613a-3dc8-4cbe-8e54-8b6ffc9d575d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:54:57.118136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:54:57.119377 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:54:57.119974 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:54:57.120528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] PATH_INFO: `/v3/projects/63297fdcd97b4e2181d3d2fcde70c516/users/7c36cbb15c5c49d0a1ea96c6cc0b9ab8/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:54:57.123869 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63297fdcd97b4e2181d3d2fcde70c516, user_id=7c36cbb15c5c49d0a1ea96c6cc0b9ab8, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:54:57.128950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:54:57.151327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1425/2853] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:54:57 2022] PUT /identity/v3/projects/63297fdcd97b4e2181d3d2fcde70c516/users/7c36cbb15c5c49d0a1ea96c6cc0b9ab8/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:55:05.235521 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced86b6d-f6b6-4b98-a50a-767c608ac1d6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:05.236478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced86b6d-f6b6-4b98-a50a-767c608ac1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:05.236478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced86b6d-f6b6-4b98-a50a-767c608ac1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:05.276496 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ced86b6d-f6b6-4b98-a50a-767c608ac1d6 None None] MFA Rules not processed for user `7c36cbb15c5c49d0a1ea96c6cc0b9ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:55:05.277584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ced86b6d-f6b6-4b98-a50a-767c608ac1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:05.387151 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:55:05.403300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1429/2854] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:55:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:55:05.423099 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:55:05.425904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:05.562741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fea6b-7c0b-4837-a33a-fb3a5018472d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:55:05.571180 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903b514-4d71-4adb-95f1-7511acd4279f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:55:05.572208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b514-4d71-4adb-95f1-7511acd4279f service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:05.572628 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b514-4d71-4adb-95f1-7511acd4279f service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:05.573026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903b514-4d71-4adb-95f1-7511acd4279f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:05.658397 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903b514-4d71-4adb-95f1-7511acd4279f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:55:05.660922 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903b514-4d71-4adb-95f1-7511acd4279f service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:55:05.673359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1426/2855] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:55:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:55:10.967221 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba0042-5aa4-4954-9439-433c44fb3672 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:10.968232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba0042-5aa4-4954-9439-433c44fb3672 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:10.968232 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba0042-5aa4-4954-9439-433c44fb3672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:10.991082 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b4ba0042-5aa4-4954-9439-433c44fb3672 None None] MFA Rules not processed for user `48ae2b062eb644059edfa65d61f60f78`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:55:10.992945 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b4ba0042-5aa4-4954-9439-433c44fb3672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:11.060743 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1430/2856] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:55:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:55:15.667931 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:55:21.053585 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=28 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:55:25.463581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613089-9946-45a0-8d80-8dc12d569b51 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:25.464450 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613089-9946-45a0-8d80-8dc12d569b51 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:25.464610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613089-9946-45a0-8d80-8dc12d569b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:25.487849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8c613089-9946-45a0-8d80-8dc12d569b51 None None] MFA Rules not processed for user `171b9c24597545a1a5c8388849ef5819`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:55:25.489352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8c613089-9946-45a0-8d80-8dc12d569b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:25.497717 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1427/2857] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:55:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:55:34.500913 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347bfb4-1dff-408f-9ae3-7a30d87feabf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:34.501571 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347bfb4-1dff-408f-9ae3-7a30d87feabf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:34.501669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347bfb4-1dff-408f-9ae3-7a30d87feabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:34.586824 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e347bfb4-1dff-408f-9ae3-7a30d87feabf None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:55:34.588190 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e347bfb4-1dff-408f-9ae3-7a30d87feabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:34.667749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1431/2858] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 31 12:55:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:55:34.728375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c613089-9946-45a0-8d80-8dc12d569b51 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:55:34.729968 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8c613089-9946-45a0-8d80-8dc12d569b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:34.839806 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c613089-9946-45a0-8d80-8dc12d569b51 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:55:34.846096 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d82ff3-3a2e-4802-b367-fce165c500ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:55:34.846905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d82ff3-3a2e-4802-b367-fce165c500ed service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:34.847215 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d82ff3-3a2e-4802-b367-fce165c500ed service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:34.847582 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d82ff3-3a2e-4802-b367-fce165c500ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:34.858381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d82ff3-3a2e-4802-b367-fce165c500ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:55:34.860568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d82ff3-3a2e-4802-b367-fce165c500ed service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:55:34.874165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1428/2859] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:55:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 12:55:44.663844 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=44 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:55:44.866826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=44 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:55:47.977916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347bfb4-1dff-408f-9ae3-7a30d87feabf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:55:48.051449 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347bfb4-1dff-408f-9ae3-7a30d87feabf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:55:48.056546 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4c289-f667-4ff3-84cc-6a55e039132d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:55:48.057181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4c289-f667-4ff3-84cc-6a55e039132d service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:48.057457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4c289-f667-4ff3-84cc-6a55e039132d service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:48.057740 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4c289-f667-4ff3-84cc-6a55e039132d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:48.083074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4c289-f667-4ff3-84cc-6a55e039132d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:55:48.084713 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4c289-f667-4ff3-84cc-6a55e039132d service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:55:48.111207 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1432/2860] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:55:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:55:54.228856 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef41dea-73be-4582-90fd-8e614958faa3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:54.229458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef41dea-73be-4582-90fd-8e614958faa3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:54.229458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef41dea-73be-4582-90fd-8e614958faa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:54.252458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9ef41dea-73be-4582-90fd-8e614958faa3 None None] MFA Rules not processed for user `ce1444d8c66045389eac37f8a7309378`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:55:54.253744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9ef41dea-73be-4582-90fd-8e614958faa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:54.262136 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1429/2861] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 31 12:55:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:55:54.278351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4c289-f667-4ff3-84cc-6a55e039132d service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:55:54.290642 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4c289-f667-4ff3-84cc-6a55e039132d service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:55:54.296553 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '981e9da7f2774c90a667fab35ecab64b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:55:54.297505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:54.297887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:54.298434 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:54.308679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:55:54.310378 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:55:54.320013 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1433/2862] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 31 12:55:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 12:55:56.561127 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef41dea-73be-4582-90fd-8e614958faa3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:55:56.563140 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9ef41dea-73be-4582-90fd-8e614958faa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:56.652254 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef41dea-73be-4582-90fd-8e614958faa3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:55:56.667710 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:55:56.668719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:56.669382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:56.669642 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:56.672859 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:56.676876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:55:56.678644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:55:56.709186 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:55:56.712275 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:55:56.741197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1430/2863] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:55:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:55:56.807009 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dc1d5-30e4-4b4f-bc39-d2671895bfbc service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:55:56.816415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:55:56.817425 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:56.817959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:56.818543 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] PATH_INFO: `/v3/users/48ae2b062eb644059edfa65d61f60f78` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:56.820853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ae2b062eb644059edfa65d61f60f78)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:55:56.825062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:55:56.909169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ae2b062eb644059edfa65d61f60f78'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:56.910279 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ae2b062eb644059edfa65d61f60f78'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:56.922861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ae2b062eb644059edfa65d61f60f78'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:56.923820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ae2b062eb644059edfa65d61f60f78'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:56.946975 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ae2b062eb644059edfa65d61f60f78'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:56.979465 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1434/2864] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:55:56 2022] DELETE /identity/v3/users/48ae2b062eb644059edfa65d61f60f78 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:55:57.248477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:55:57.280478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a31a59-90e6-47a3-aee7-7d8c534b089d service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:55:57.378285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:55:57.380762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:55:57.381148 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:55:57.381506 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] PATH_INFO: `/v3/projects/e025dc367b3a4349b1f708b7b3966d43` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:55:57.383000 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e025dc367b3a4349b1f708b7b3966d43)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:55:57.386811 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:55:57.465159 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e025dc367b3a4349b1f708b7b3966d43', 'request_id': 'req-e2c890e4-d2ec-491f-867f-3dd4cdb79255'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:57.476572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e025dc367b3a4349b1f708b7b3966d43', 'request_id': 'req-e2c890e4-d2ec-491f-867f-3dd4cdb79255'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:57.491303 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e025dc367b3a4349b1f708b7b3966d43', 'request_id': 'req-e2c890e4-d2ec-491f-867f-3dd4cdb79255'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:57.519952 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] The token cache is being invalidate because project e025dc367b3a4349b1f708b7b3966d43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:55:57.520526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:57.521363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:55:57.523218 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1431/2865] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:55:57 2022] DELETE /identity/v3/projects/e025dc367b3a4349b1f708b7b3966d43 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:56:00.316236 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:56:00.317753 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:56:00.334412 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:56:00.398122 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1db1fc-2f64-4891-a54d-b2d22fb09450 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:56:00.458043 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c54ac7-6850-4603-8b9e-435b1b1c3115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:56:00.458659 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54ac7-6850-4603-8b9e-435b1b1c3115 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:56:00.458927 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54ac7-6850-4603-8b9e-435b1b1c3115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:56:00.459223 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54ac7-6850-4603-8b9e-435b1b1c3115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:56:00.461109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-31c54ac7-6850-4603-8b9e-435b1b1c3115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:56:00.471446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c54ac7-6850-4603-8b9e-435b1b1c3115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:56:00.472985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c54ac7-6850-4603-8b9e-435b1b1c3115 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:56:00.482263 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1435/2866] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:56:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:56:07.514223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:56:10.478104 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=42 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:57:15.915267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:57:15.916937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:57:16.050266 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c890e4-d2ec-491f-867f-3dd4cdb79255 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:57:16.055673 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5631f-db5a-47b3-b406-52f7512ea79e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:57:16.056291 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5631f-db5a-47b3-b406-52f7512ea79e service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:57:16.056547 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5631f-db5a-47b3-b406-52f7512ea79e service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:57:16.056821 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5631f-db5a-47b3-b406-52f7512ea79e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:57:16.058758 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6df5631f-db5a-47b3-b406-52f7512ea79e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:57:16.071061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5631f-db5a-47b3-b406-52f7512ea79e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:57:16.072741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5631f-db5a-47b3-b406-52f7512ea79e service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:57:16.083684 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1432/2867] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:57:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:57:26.078745 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=49 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:57:33.664787 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d03af-edb9-4c21-bfe4-0bee8da1ff17 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:57:33.665457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d03af-edb9-4c21-bfe4-0bee8da1ff17 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:57:33.665457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d03af-edb9-4c21-bfe4-0bee8da1ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:57:33.688119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ea0d03af-edb9-4c21-bfe4-0bee8da1ff17 None None] MFA Rules not processed for user `454a193b26684746bc8c9278a1f1f175`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:57:33.689461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ea0d03af-edb9-4c21-bfe4-0bee8da1ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:57:33.758374 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1436/2868] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:57:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:57:43.750444 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=25 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:57:55.002797 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0985fc-3a15-4d94-930d-96aa03512f61 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:57:55.003558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0985fc-3a15-4d94-930d-96aa03512f61 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:57:55.003976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0985fc-3a15-4d94-930d-96aa03512f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:57:55.035618 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8a0985fc-3a15-4d94-930d-96aa03512f61 None None] MFA Rules not processed for user `171b9c24597545a1a5c8388849ef5819`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:57:55.037666 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a0985fc-3a15-4d94-930d-96aa03512f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:57:55.111949 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1433/2869] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:57:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:58:05.104045 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=25 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:58:58.084467 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d03af-edb9-4c21-bfe4-0bee8da1ff17 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:58.087031 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ea0d03af-edb9-4c21-bfe4-0bee8da1ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:58:58.193290 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d03af-edb9-4c21-bfe4-0bee8da1ff17 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:58.200978 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:58.202035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:58.202534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:58.203017 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] PATH_INFO: `/v3/users/454a193b26684746bc8c9278a1f1f175` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:58.205194 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=454a193b26684746bc8c9278a1f1f175)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:58.209355 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:58.283594 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454a193b26684746bc8c9278a1f1f175'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.284208 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454a193b26684746bc8c9278a1f1f175'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.295055 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454a193b26684746bc8c9278a1f1f175'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.296032 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454a193b26684746bc8c9278a1f1f175'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.312700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcb54f35-7f42-4d71-b319-66335cde8e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454a193b26684746bc8c9278a1f1f175'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.327882 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1437/2870] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:58:58 2022] DELETE /identity/v3/users/454a193b26684746bc8c9278a1f1f175 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:58:58.543439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0985fc-3a15-4d94-930d-96aa03512f61 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:58.560138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0985fc-3a15-4d94-930d-96aa03512f61 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:58.642884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:58.643589 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:58.643867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:58.644138 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] PATH_INFO: `/v3/projects/23ff9f71e16d460781dd5889d784b4a3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:58.645777 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ff9f71e16d460781dd5889d784b4a3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:58.647583 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:58.712725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ff9f71e16d460781dd5889d784b4a3', 'request_id': 'req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.723164 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ff9f71e16d460781dd5889d784b4a3', 'request_id': 'req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.734820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ff9f71e16d460781dd5889d784b4a3', 'request_id': 'req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.755484 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] The token cache is being invalidate because project 23ff9f71e16d460781dd5889d784b4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:58:58.755957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.757047 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1bf385b3-a51b-4ac6-b0c0-9693491d19e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:58:58.759615 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1434/2871] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:58:58 2022] DELETE /identity/v3/projects/23ff9f71e16d460781dd5889d784b4a3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 12:58:58.767453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369648e-aa72-40e3-a25a-17acc6cacf55 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:58.767931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369648e-aa72-40e3-a25a-17acc6cacf55 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:58.768376 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369648e-aa72-40e3-a25a-17acc6cacf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:58.800929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f369648e-aa72-40e3-a25a-17acc6cacf55 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:58:58.802127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f369648e-aa72-40e3-a25a-17acc6cacf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:58:58.886035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1438/2872] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:58:58.895213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e30a5-7052-4f61-a815-270519a22592 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:58.895725 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e30a5-7052-4f61-a815-270519a22592 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:58.896807 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e30a5-7052-4f61-a815-270519a22592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:58.920207 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-793e30a5-7052-4f61-a815-270519a22592 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:58:58.922737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-793e30a5-7052-4f61-a815-270519a22592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:58:58.936260 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1435/2873] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:58:58.941382 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369648e-aa72-40e3-a25a-17acc6cacf55 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:58.959337 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369648e-aa72-40e3-a25a-17acc6cacf55 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:58.967903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:58.968959 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:58.969464 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:58.969934 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:58.971173 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:58:58.971752 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:58.974883 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:58.997972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1439/2874] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:58:59.003605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e30a5-7052-4f61-a815-270519a22592 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:59.014178 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e30a5-7052-4f61-a815-270519a22592 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:59.019805 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a7b9d-675a-4140-94ff-6e106dc008f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:59.020770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a7b9d-675a-4140-94ff-6e106dc008f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:59.020770 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a7b9d-675a-4140-94ff-6e106dc008f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:59.021075 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a7b9d-675a-4140-94ff-6e106dc008f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:59.021657 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a7b9d-675a-4140-94ff-6e106dc008f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:59.023645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a7b9d-675a-4140-94ff-6e106dc008f7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:59.060633 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1436/2875] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:58:59 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:58:59.065226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:59.076377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a80deb-6818-4232-92f8-92377de5a985 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:59.080793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:58:59.154278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c411fb-ba4e-48ee-8e14-69d0f8eccf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:59.155018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c411fb-ba4e-48ee-8e14-69d0f8eccf8d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:59.155334 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c411fb-ba4e-48ee-8e14-69d0f8eccf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:59.155675 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c411fb-ba4e-48ee-8e14-69d0f8eccf8d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:59.156361 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c411fb-ba4e-48ee-8e14-69d0f8eccf8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:59.158770 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c411fb-ba4e-48ee-8e14-69d0f8eccf8d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:59.188497 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1440/2876] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:58:59 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:58:59.193703 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a7b9d-675a-4140-94ff-6e106dc008f7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:59.211958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a7b9d-675a-4140-94ff-6e106dc008f7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:59.219875 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8283a-d9e8-4764-9ba0-a1c9412bde5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:59.220877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8283a-d9e8-4764-9ba0-a1c9412bde5b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:59.221338 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8283a-d9e8-4764-9ba0-a1c9412bde5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:59.221782 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8283a-d9e8-4764-9ba0-a1c9412bde5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:59.223109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8283a-d9e8-4764-9ba0-a1c9412bde5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:59.225867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8283a-d9e8-4764-9ba0-a1c9412bde5b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:59.248526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1437/2877] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:58:59.253348 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c411fb-ba4e-48ee-8e14-69d0f8eccf8d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:59.265672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c411fb-ba4e-48ee-8e14-69d0f8eccf8d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:59.271688 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bc90c-e7c6-4052-88c0-a401b4772374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:59.272326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bc90c-e7c6-4052-88c0-a401b4772374 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:59.272605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bc90c-e7c6-4052-88c0-a401b4772374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:59.272871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bc90c-e7c6-4052-88c0-a401b4772374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:59.273529 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bc90c-e7c6-4052-88c0-a401b4772374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:59.275180 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bc90c-e7c6-4052-88c0-a401b4772374 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:59.289915 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1441/2878] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:58:59.294542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8283a-d9e8-4764-9ba0-a1c9412bde5b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:59.305796 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8283a-d9e8-4764-9ba0-a1c9412bde5b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:59.311361 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:59.312108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:59.312422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:59.312731 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] PATH_INFO: `/v3/projects/4b56de3a35d74eee8d4dc20b6ab9ab65/users/9dcc8fb1b37e40e899c66d42278ef144/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:59.330309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b56de3a35d74eee8d4dc20b6ab9ab65, user_id=9dcc8fb1b37e40e899c66d42278ef144, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:59.333182 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:59.348898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1438/2879] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:58:59 2022] PUT /identity/v3/projects/4b56de3a35d74eee8d4dc20b6ab9ab65/users/9dcc8fb1b37e40e899c66d42278ef144/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:58:59.360016 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9240c34-dfbd-431f-85c7-9c115a404a4d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:59.360380 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9240c34-dfbd-431f-85c7-9c115a404a4d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:59.360719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9240c34-dfbd-431f-85c7-9c115a404a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:59.416524 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b9240c34-dfbd-431f-85c7-9c115a404a4d None None] MFA Rules not processed for user `9dcc8fb1b37e40e899c66d42278ef144`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:58:59.417972 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b9240c34-dfbd-431f-85c7-9c115a404a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:58:59.532347 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1442/2880] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:58:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:58:59.545226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:59.547577 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:58:59.585769 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:58:59.642256 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd43c9a-6067-49ce-b3fc-2ecefbfcffc0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:59.649181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:59.649818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:59.650226 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:59.650449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:59.659889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:59.661620 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:59.673160 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1439/2881] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:58:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:58:59.772854 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9240c34-dfbd-431f-85c7-9c115a404a4d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:58:59.774439 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b9240c34-dfbd-431f-85c7-9c115a404a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:58:59.872598 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9240c34-dfbd-431f-85c7-9c115a404a4d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:58:59.881109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:58:59.881872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:58:59.882213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:58:59.882509 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:58:59.884821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:58:59.902147 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:58:59.903672 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:58:59.917986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1443/2882] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:59:00.006998 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:00.008610 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:00.022350 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4335590-6247-45db-9a9b-060c0929838d service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:00.026613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fb051-0604-4df8-9ae8-05108d5d0d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:00.027287 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb051-0604-4df8-9ae8-05108d5d0d8f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:00.027558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb051-0604-4df8-9ae8-05108d5d0d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:00.027832 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb051-0604-4df8-9ae8-05108d5d0d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:00.029818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ac0fb051-0604-4df8-9ae8-05108d5d0d8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:00.046052 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fb051-0604-4df8-9ae8-05108d5d0d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:00.047604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fb051-0604-4df8-9ae8-05108d5d0d8f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:00.062125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1440/2883] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:59:00.151744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:00.176118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:00.231059 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234141ab-338c-47b0-a245-c9adc0c39e36 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:00.236227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:00.236892 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:00.237171 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:00.237443 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] PATH_INFO: `/v3/users/9dcc8fb1b37e40e899c66d42278ef144` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:00.238823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dcc8fb1b37e40e899c66d42278ef144)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:00.241041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:00.308582 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dcc8fb1b37e40e899c66d42278ef144'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:00.309308 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dcc8fb1b37e40e899c66d42278ef144'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:00.319453 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dcc8fb1b37e40e899c66d42278ef144'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:00.320076 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dcc8fb1b37e40e899c66d42278ef144'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:00.337515 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dcc8fb1b37e40e899c66d42278ef144'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:00.358980 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1444/2884] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:59:00 2022] DELETE /identity/v3/users/9dcc8fb1b37e40e899c66d42278ef144 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:00.364231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd633-b94f-477d-8635-ac40897ca867 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:00.364563 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd633-b94f-477d-8635-ac40897ca867 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:00.364874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd633-b94f-477d-8635-ac40897ca867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:00.386756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-000cd633-b94f-477d-8635-ac40897ca867 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:00.387897 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-000cd633-b94f-477d-8635-ac40897ca867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:00.461440 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1441/2885] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:00.472167 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:00.474520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:00.547120 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c8279-177c-4014-8068-f3e2d7eba243 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:00.552005 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ab44b-d223-4a90-8329-6868e0ca82ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:00.552693 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ab44b-d223-4a90-8329-6868e0ca82ae service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:00.553008 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ab44b-d223-4a90-8329-6868e0ca82ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:00.553319 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ab44b-d223-4a90-8329-6868e0ca82ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:00.564745 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ab44b-d223-4a90-8329-6868e0ca82ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:00.566255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ab44b-d223-4a90-8329-6868e0ca82ae service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:00.577609 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1445/2886] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:59:00.980237 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cd633-b94f-477d-8635-ac40897ca867 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:00.982143 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-000cd633-b94f-477d-8635-ac40897ca867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:01.002730 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cd633-b94f-477d-8635-ac40897ca867 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:01.011385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:01.012454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.013704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.013704 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] PATH_INFO: `/v3/projects/4b56de3a35d74eee8d4dc20b6ab9ab65` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.015681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b56de3a35d74eee8d4dc20b6ab9ab65)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:01.018718 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:01.083729 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b56de3a35d74eee8d4dc20b6ab9ab65', 'request_id': 'req-fe6b183f-ca39-4f02-a0cf-6701c9b38150'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:01.090744 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b56de3a35d74eee8d4dc20b6ab9ab65', 'request_id': 'req-fe6b183f-ca39-4f02-a0cf-6701c9b38150'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:01.098381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b56de3a35d74eee8d4dc20b6ab9ab65', 'request_id': 'req-fe6b183f-ca39-4f02-a0cf-6701c9b38150'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:01.115655 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] The token cache is being invalidate because project 4b56de3a35d74eee8d4dc20b6ab9ab65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:59:01.116017 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:01.116765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe6b183f-ca39-4f02-a0cf-6701c9b38150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:01.118515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1442/2887] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:59:00 2022] DELETE /identity/v3/projects/4b56de3a35d74eee8d4dc20b6ab9ab65 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:01.123989 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b59a6-c25e-4d27-88ee-a8a49c016ad5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.124373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b59a6-c25e-4d27-88ee-a8a49c016ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.124717 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b59a6-c25e-4d27-88ee-a8a49c016ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.150998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-077b59a6-c25e-4d27-88ee-a8a49c016ad5 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:01.153169 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-077b59a6-c25e-4d27-88ee-a8a49c016ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:01.209042 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:01.262764 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1446/2888] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:01.273329 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d006e57-c561-4fb2-a880-e3710b85d139 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.273676 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d006e57-c561-4fb2-a880-e3710b85d139 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.273980 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d006e57-c561-4fb2-a880-e3710b85d139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.298803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4d006e57-c561-4fb2-a880-e3710b85d139 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:01.300436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4d006e57-c561-4fb2-a880-e3710b85d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:01.310242 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1443/2889] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:01.315143 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b59a6-c25e-4d27-88ee-a8a49c016ad5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:01.328184 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b59a6-c25e-4d27-88ee-a8a49c016ad5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:01.334877 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:01.335746 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.336195 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.336655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.337711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:59:01.338066 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:01.340458 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:01.357644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1447/2890] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:59:01.365460 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d006e57-c561-4fb2-a880-e3710b85d139 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:01.376438 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d006e57-c561-4fb2-a880-e3710b85d139 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:01.382566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec6827-1b0b-492f-b317-32dd24fb8030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:01.383646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec6827-1b0b-492f-b317-32dd24fb8030 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.383789 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec6827-1b0b-492f-b317-32dd24fb8030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.384058 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec6827-1b0b-492f-b317-32dd24fb8030 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.384723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec6827-1b0b-492f-b317-32dd24fb8030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:01.386866 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec6827-1b0b-492f-b317-32dd24fb8030 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:01.406756 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:01.439385 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1444/2891] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:59:01 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:01.441300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:01.455001 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b427e-efbb-4e51-a1b3-d5e4045aaa55 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:01.544851 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375facda-adae-44b4-917b-bbe702eaf5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:01.547035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375facda-adae-44b4-917b-bbe702eaf5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.547035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375facda-adae-44b4-917b-bbe702eaf5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.547035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375facda-adae-44b4-917b-bbe702eaf5cf admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.547983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375facda-adae-44b4-917b-bbe702eaf5cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:01.550036 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375facda-adae-44b4-917b-bbe702eaf5cf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:01.584102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1448/2892] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:59:01 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:59:01.590643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec6827-1b0b-492f-b317-32dd24fb8030 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:01.603534 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec6827-1b0b-492f-b317-32dd24fb8030 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:01.608873 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f51895-41fd-47ba-b08c-2d7637befda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:01.609742 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51895-41fd-47ba-b08c-2d7637befda6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.610173 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51895-41fd-47ba-b08c-2d7637befda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.610558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51895-41fd-47ba-b08c-2d7637befda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.611316 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f51895-41fd-47ba-b08c-2d7637befda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:01.613264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f51895-41fd-47ba-b08c-2d7637befda6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:01.625976 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1445/2893] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:01.630653 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375facda-adae-44b4-917b-bbe702eaf5cf admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:01.642697 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375facda-adae-44b4-917b-bbe702eaf5cf admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:01.649823 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800e256-bf24-4f7a-b61e-1dd12b8d97ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:01.650669 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e256-bf24-4f7a-b61e-1dd12b8d97ed admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.651049 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e256-bf24-4f7a-b61e-1dd12b8d97ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.651513 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e256-bf24-4f7a-b61e-1dd12b8d97ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.652452 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800e256-bf24-4f7a-b61e-1dd12b8d97ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:01.654405 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800e256-bf24-4f7a-b61e-1dd12b8d97ed admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:01.669056 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1449/2894] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:59:01.674359 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f51895-41fd-47ba-b08c-2d7637befda6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:01.686108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f51895-41fd-47ba-b08c-2d7637befda6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:01.691880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f730a-6e44-40ae-81e8-b727e7a6ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:01.693406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f730a-6e44-40ae-81e8-b727e7a6ddbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.693733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f730a-6e44-40ae-81e8-b727e7a6ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.694054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f730a-6e44-40ae-81e8-b727e7a6ddbd admin admin] PATH_INFO: `/v3/projects/a57261b580924c478f62f881ef10a937/users/6d0bd6a3f2054679b82bcac2c5afc13e/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.712464 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f730a-6e44-40ae-81e8-b727e7a6ddbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a57261b580924c478f62f881ef10a937, user_id=6d0bd6a3f2054679b82bcac2c5afc13e, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:01.715619 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f730a-6e44-40ae-81e8-b727e7a6ddbd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:01.732368 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1446/2895] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:59:01 2022] PUT /identity/v3/projects/a57261b580924c478f62f881ef10a937/users/6d0bd6a3f2054679b82bcac2c5afc13e/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:01.744132 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d30b-ba85-45c2-82cb-9f63f22da12b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.744577 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d30b-ba85-45c2-82cb-9f63f22da12b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.744967 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d30b-ba85-45c2-82cb-9f63f22da12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:01.778876 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-67e2d30b-ba85-45c2-82cb-9f63f22da12b None None] MFA Rules not processed for user `6d0bd6a3f2054679b82bcac2c5afc13e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:01.780395 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-67e2d30b-ba85-45c2-82cb-9f63f22da12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:01.867671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1450/2896] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:59:01.875823 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f730a-6e44-40ae-81e8-b727e7a6ddbd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:01.980191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f730a-6e44-40ae-81e8-b727e7a6ddbd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:01.996804 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdf6f4-e7aa-4843-918f-fba4f0f27744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:01.998118 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf6f4-e7aa-4843-918f-fba4f0f27744 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:01.998613 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf6f4-e7aa-4843-918f-fba4f0f27744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:01.999102 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf6f4-e7aa-4843-918f-fba4f0f27744 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:02.000694 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdf6f4-e7aa-4843-918f-fba4f0f27744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:02.004367 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdf6f4-e7aa-4843-918f-fba4f0f27744 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:02.047165 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1447/2897] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:59:01 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:02.053778 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d30b-ba85-45c2-82cb-9f63f22da12b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:02.068227 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d30b-ba85-45c2-82cb-9f63f22da12b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:02.116579 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:02.165477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae729ee-232a-4f87-866b-d421f953aeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:02.166476 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae729ee-232a-4f87-866b-d421f953aeee admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:02.166966 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae729ee-232a-4f87-866b-d421f953aeee admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:02.167499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae729ee-232a-4f87-866b-d421f953aeee admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:02.168720 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae729ee-232a-4f87-866b-d421f953aeee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:02.172729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae729ee-232a-4f87-866b-d421f953aeee admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:02.211559 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1451/2898] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:59:02 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:02.216772 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdf6f4-e7aa-4843-918f-fba4f0f27744 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:02.233062 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdf6f4-e7aa-4843-918f-fba4f0f27744 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:02.241631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab371a0-5674-45aa-b412-f9b0f0aa4cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:02.242337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab371a0-5674-45aa-b412-f9b0f0aa4cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:02.242609 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab371a0-5674-45aa-b412-f9b0f0aa4cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:02.242887 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab371a0-5674-45aa-b412-f9b0f0aa4cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:02.243605 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab371a0-5674-45aa-b412-f9b0f0aa4cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:02.245285 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab371a0-5674-45aa-b412-f9b0f0aa4cbb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:02.259723 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1448/2899] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:02.264658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae729ee-232a-4f87-866b-d421f953aeee admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:02.279282 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae729ee-232a-4f87-866b-d421f953aeee admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:02.285477 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0d7e-85fe-4880-9b33-5998a201a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:02.286872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0d7e-85fe-4880-9b33-5998a201a7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:02.287046 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0d7e-85fe-4880-9b33-5998a201a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:02.287474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0d7e-85fe-4880-9b33-5998a201a7c2 admin admin] PATH_INFO: `/v3/projects/8a987d0c0cdb4f64866607b84aa31a0f/users/6eb63484bfa94e79a77a0766526765d3/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:02.306996 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b0d7e-85fe-4880-9b33-5998a201a7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a987d0c0cdb4f64866607b84aa31a0f, user_id=6eb63484bfa94e79a77a0766526765d3, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:02.310555 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b0d7e-85fe-4880-9b33-5998a201a7c2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:02.330185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1452/2900] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:59:02 2022] PUT /identity/v3/projects/8a987d0c0cdb4f64866607b84aa31a0f/users/6eb63484bfa94e79a77a0766526765d3/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:02.335264 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab371a0-5674-45aa-b412-f9b0f0aa4cbb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:02.349842 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab371a0-5674-45aa-b412-f9b0f0aa4cbb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:02.442665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab6d06-bf32-4e55-bdbc-84be658f82a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:02.443393 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6d06-bf32-4e55-bdbc-84be658f82a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:02.443753 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6d06-bf32-4e55-bdbc-84be658f82a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:02.444115 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6d06-bf32-4e55-bdbc-84be658f82a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:02.444917 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab6d06-bf32-4e55-bdbc-84be658f82a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:02.446667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab6d06-bf32-4e55-bdbc-84be658f82a3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:02.458809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1449/2901] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:02.463432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0d7e-85fe-4880-9b33-5998a201a7c2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:02.478606 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0d7e-85fe-4880-9b33-5998a201a7c2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:02.483581 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:02.484268 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:02.484545 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:02.484818 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] PATH_INFO: `/v3/projects/8a987d0c0cdb4f64866607b84aa31a0f/users/6eb63484bfa94e79a77a0766526765d3/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:02.486841 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a987d0c0cdb4f64866607b84aa31a0f, user_id=6eb63484bfa94e79a77a0766526765d3, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:02.490404 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:02.503658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1453/2902] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:59:02 2022] PUT /identity/v3/projects/8a987d0c0cdb4f64866607b84aa31a0f/users/6eb63484bfa94e79a77a0766526765d3/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:02.516104 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c3e6a-9dbd-411b-a8d2-2d2cb1d0c32e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:02.516509 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c3e6a-9dbd-411b-a8d2-2d2cb1d0c32e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:02.516903 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c3e6a-9dbd-411b-a8d2-2d2cb1d0c32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:02.551570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c90c3e6a-9dbd-411b-a8d2-2d2cb1d0c32e None None] MFA Rules not processed for user `6eb63484bfa94e79a77a0766526765d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:02.552979 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c90c3e6a-9dbd-411b-a8d2-2d2cb1d0c32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:02.574893 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:02.654784 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1450/2903] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:02.668219 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:02.670102 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:02.746119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ed89-acc6-4254-8632-7fc8e348c1c8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:02.752871 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de764a3-8a21-4422-abb8-bb29d9ae7b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:02.753875 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de764a3-8a21-4422-abb8-bb29d9ae7b40 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:02.754359 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de764a3-8a21-4422-abb8-bb29d9ae7b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:02.754762 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de764a3-8a21-4422-abb8-bb29d9ae7b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:02.768922 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de764a3-8a21-4422-abb8-bb29d9ae7b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:02.771061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de764a3-8a21-4422-abb8-bb29d9ae7b40 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:02.784817 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1454/2904] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:59:02.908030 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c3e6a-9dbd-411b-a8d2-2d2cb1d0c32e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:02.919668 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c3e6a-9dbd-411b-a8d2-2d2cb1d0c32e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:02.924153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c99dc-6388-43dc-8662-63009853a56c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:02.925050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c99dc-6388-43dc-8662-63009853a56c service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:02.925334 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c99dc-6388-43dc-8662-63009853a56c service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:02.925685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c99dc-6388-43dc-8662-63009853a56c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:03.005774 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c99dc-6388-43dc-8662-63009853a56c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:03.007581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c99dc-6388-43dc-8662-63009853a56c service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:03.018262 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1451/2905] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:59:12.777861 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=54,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:13.013565 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=49 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:13.421106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de764a3-8a21-4422-abb8-bb29d9ae7b40 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:13.557937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de764a3-8a21-4422-abb8-bb29d9ae7b40 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:13.563416 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:13.564083 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:13.564373 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:13.564648 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] PATH_INFO: `/v3/users/6d0bd6a3f2054679b82bcac2c5afc13e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:13.565944 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0bd6a3f2054679b82bcac2c5afc13e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:13.568028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:13.626837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0bd6a3f2054679b82bcac2c5afc13e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:13.627627 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0bd6a3f2054679b82bcac2c5afc13e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:13.637711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0bd6a3f2054679b82bcac2c5afc13e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:13.638617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0bd6a3f2054679b82bcac2c5afc13e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:13.654110 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0bd6a3f2054679b82bcac2c5afc13e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:13.668970 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1455/2906] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:59:13 2022] DELETE /identity/v3/users/6d0bd6a3f2054679b82bcac2c5afc13e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:13.674309 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5aaf0-584a-403f-a054-15039afbc423 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:13.674594 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5aaf0-584a-403f-a054-15039afbc423 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:13.674874 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5aaf0-584a-403f-a054-15039afbc423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:13.696388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-30d5aaf0-584a-403f-a054-15039afbc423 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:13.697664 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-30d5aaf0-584a-403f-a054-15039afbc423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:13.771857 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1452/2907] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:59:13.782806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:13.784427 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:13.865655 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c74511-ded1-4760-a9fb-6b77c0a4b804 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:13.869790 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2de9c-22f0-465e-856c-62f05bf18c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:13.870494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2de9c-22f0-465e-856c-62f05bf18c29 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:13.871333 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2de9c-22f0-465e-856c-62f05bf18c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:13.871705 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2de9c-22f0-465e-856c-62f05bf18c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:13.885599 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2de9c-22f0-465e-856c-62f05bf18c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:13.887246 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2de9c-22f0-465e-856c-62f05bf18c29 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:13.893185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:13.905160 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1456/2908] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:59:14.296479 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5aaf0-584a-403f-a054-15039afbc423 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:14.298174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-30d5aaf0-584a-403f-a054-15039afbc423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:14.313391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5aaf0-584a-403f-a054-15039afbc423 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:14.320388 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:14.321053 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:14.321333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:14.321606 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] PATH_INFO: `/v3/projects/a57261b580924c478f62f881ef10a937` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:14.322934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a57261b580924c478f62f881ef10a937)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:14.324966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:14.384629 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a57261b580924c478f62f881ef10a937', 'request_id': 'req-4aa74f57-ff07-4532-8592-bdde368350c0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.392297 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a57261b580924c478f62f881ef10a937', 'request_id': 'req-4aa74f57-ff07-4532-8592-bdde368350c0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.400711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a57261b580924c478f62f881ef10a937', 'request_id': 'req-4aa74f57-ff07-4532-8592-bdde368350c0'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.420227 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] The token cache is being invalidate because project a57261b580924c478f62f881ef10a937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:59:14.420764 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.421773 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.424395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1453/2909] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:59:14 2022] DELETE /identity/v3/projects/a57261b580924c478f62f881ef10a937 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:14.430417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2de9c-22f0-465e-856c-62f05bf18c29 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:14.442140 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2de9c-22f0-465e-856c-62f05bf18c29 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:14.516584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:14.517390 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:14.517760 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:14.518191 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] PATH_INFO: `/v3/users/6eb63484bfa94e79a77a0766526765d3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:14.519732 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb63484bfa94e79a77a0766526765d3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:14.522300 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:14.584837 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb63484bfa94e79a77a0766526765d3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.585695 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb63484bfa94e79a77a0766526765d3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.596754 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb63484bfa94e79a77a0766526765d3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.597684 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb63484bfa94e79a77a0766526765d3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.613829 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da4adea-ae6b-49a2-bc74-9b9718a18b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb63484bfa94e79a77a0766526765d3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:14.631700 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1457/2910] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:59:14 2022] DELETE /identity/v3/users/6eb63484bfa94e79a77a0766526765d3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:14.996965 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:14.998696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:15.026488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa74f57-ff07-4532-8592-bdde368350c0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:15.101890 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:15.102558 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.102834 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.103121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] PATH_INFO: `/v3/projects/8a987d0c0cdb4f64866607b84aa31a0f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.104380 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a987d0c0cdb4f64866607b84aa31a0f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:15.106682 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:15.144898 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:15.169855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a987d0c0cdb4f64866607b84aa31a0f', 'request_id': 'req-b8700470-f490-4062-a056-86a516fe8792'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:15.177518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a987d0c0cdb4f64866607b84aa31a0f', 'request_id': 'req-b8700470-f490-4062-a056-86a516fe8792'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:15.190778 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a987d0c0cdb4f64866607b84aa31a0f', 'request_id': 'req-b8700470-f490-4062-a056-86a516fe8792'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:15.217033 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] The token cache is being invalidate because project 8a987d0c0cdb4f64866607b84aa31a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:59:15.217553 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:15.218966 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b8700470-f490-4062-a056-86a516fe8792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:15.222059 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1454/2911] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:59:14 2022] DELETE /identity/v3/projects/8a987d0c0cdb4f64866607b84aa31a0f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:15.228821 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5f7c6-d624-4f32-a469-e3ecf78ee045 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.229430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5f7c6-d624-4f32-a469-e3ecf78ee045 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.229430 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5f7c6-d624-4f32-a469-e3ecf78ee045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.253461 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e9b5f7c6-d624-4f32-a469-e3ecf78ee045 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:15.255012 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e9b5f7c6-d624-4f32-a469-e3ecf78ee045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:15.340121 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1458/2912] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:59:15.349358 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a57b-001b-440b-911e-a0d18ee197af None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.350258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a57b-001b-440b-911e-a0d18ee197af None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.350258 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a57b-001b-440b-911e-a0d18ee197af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.374478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6061a57b-001b-440b-911e-a0d18ee197af None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:15.376281 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6061a57b-001b-440b-911e-a0d18ee197af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:15.386512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1455/2913] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:15.391930 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5f7c6-d624-4f32-a469-e3ecf78ee045 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:15.405711 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5f7c6-d624-4f32-a469-e3ecf78ee045 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:15.411962 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:15.412603 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.412878 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.413150 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.413832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:59:15.414133 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:15.415882 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:15.430857 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1459/2914] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 12:59:15.437245 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061a57b-001b-440b-911e-a0d18ee197af None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:15.450208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061a57b-001b-440b-911e-a0d18ee197af None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:15.455478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f7ca-ffdd-4638-acf0-6119415968a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:15.456158 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f7ca-ffdd-4638-acf0-6119415968a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.456433 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f7ca-ffdd-4638-acf0-6119415968a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.456700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f7ca-ffdd-4638-acf0-6119415968a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.457345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492f7ca-ffdd-4638-acf0-6119415968a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:15.459621 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492f7ca-ffdd-4638-acf0-6119415968a9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:15.492658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1456/2915] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:59:15 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 12:59:15.497869 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:15.510709 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:15.517862 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6c713-8efb-4e9a-8c60-9baf7fc11f91 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:15.605367 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a5efb-8429-4968-8e4f-a6acd3301e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:15.605964 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5efb-8429-4968-8e4f-a6acd3301e3f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.606385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5efb-8429-4968-8e4f-a6acd3301e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.606706 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5efb-8429-4968-8e4f-a6acd3301e3f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.607450 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a5efb-8429-4968-8e4f-a6acd3301e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:15.609756 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a5efb-8429-4968-8e4f-a6acd3301e3f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:15.639985 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1460/2916] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:59:15 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:15.644690 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f7ca-ffdd-4638-acf0-6119415968a9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:15.657221 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492f7ca-ffdd-4638-acf0-6119415968a9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:15.663542 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790d11-f87a-4a71-aa96-7bf327e79751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:15.664181 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790d11-f87a-4a71-aa96-7bf327e79751 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.664458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790d11-f87a-4a71-aa96-7bf327e79751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.664726 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790d11-f87a-4a71-aa96-7bf327e79751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.665523 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b790d11-f87a-4a71-aa96-7bf327e79751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:15.667097 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b790d11-f87a-4a71-aa96-7bf327e79751 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:15.680424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1457/2917] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:15.684979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a5efb-8429-4968-8e4f-a6acd3301e3f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:15.696474 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a5efb-8429-4968-8e4f-a6acd3301e3f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:15.701931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabef849-55b7-42b0-a2c9-c0eaaa60be30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:15.702569 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabef849-55b7-42b0-a2c9-c0eaaa60be30 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.702872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabef849-55b7-42b0-a2c9-c0eaaa60be30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.703139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabef849-55b7-42b0-a2c9-c0eaaa60be30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.703840 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabef849-55b7-42b0-a2c9-c0eaaa60be30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:15.705506 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabef849-55b7-42b0-a2c9-c0eaaa60be30 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:15.718358 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1461/2918] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:15.723375 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790d11-f87a-4a71-aa96-7bf327e79751 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:15.736530 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790d11-f87a-4a71-aa96-7bf327e79751 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:15.744302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4912da-b047-4eb4-9fa8-6cceb7fea2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:15.745448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4912da-b047-4eb4-9fa8-6cceb7fea2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.745959 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4912da-b047-4eb4-9fa8-6cceb7fea2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.746428 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4912da-b047-4eb4-9fa8-6cceb7fea2a9 admin admin] PATH_INFO: `/v3/projects/b25b02a1341042f8aeaf38e4f6006315/users/7668128b22af442dac5a9faac23c6ca5/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.774559 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4912da-b047-4eb4-9fa8-6cceb7fea2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25b02a1341042f8aeaf38e4f6006315, user_id=7668128b22af442dac5a9faac23c6ca5, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:15.779423 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4912da-b047-4eb4-9fa8-6cceb7fea2a9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:15.799845 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1458/2919] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:59:15 2022] PUT /identity/v3/projects/b25b02a1341042f8aeaf38e4f6006315/users/7668128b22af442dac5a9faac23c6ca5/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:15.811933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed8cb-46db-4a95-aea3-0f46ad461730 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:15.812262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed8cb-46db-4a95-aea3-0f46ad461730 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:15.812604 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed8cb-46db-4a95-aea3-0f46ad461730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:15.862086 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-805ed8cb-46db-4a95-aea3-0f46ad461730 None None] MFA Rules not processed for user `7668128b22af442dac5a9faac23c6ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:15.865624 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-805ed8cb-46db-4a95-aea3-0f46ad461730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:15.949557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1462/2920] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:15.954728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4912da-b047-4eb4-9fa8-6cceb7fea2a9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:16.031685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4912da-b047-4eb4-9fa8-6cceb7fea2a9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:16.037451 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f275f37-1c05-485a-b096-c7379fd9bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:16.038088 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f275f37-1c05-485a-b096-c7379fd9bf25 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:16.038369 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f275f37-1c05-485a-b096-c7379fd9bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:16.038645 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f275f37-1c05-485a-b096-c7379fd9bf25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:16.039349 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f275f37-1c05-485a-b096-c7379fd9bf25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:16.041340 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f275f37-1c05-485a-b096-c7379fd9bf25 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:16.074759 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1459/2921] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:59:15 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:16.079255 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ed8cb-46db-4a95-aea3-0f46ad461730 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:16.091912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ed8cb-46db-4a95-aea3-0f46ad461730 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:16.161213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097a9e8-f019-48d8-8c20-4804afb6b16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:16.162062 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a9e8-f019-48d8-8c20-4804afb6b16d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:16.162346 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a9e8-f019-48d8-8c20-4804afb6b16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:16.162620 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a9e8-f019-48d8-8c20-4804afb6b16d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:16.163417 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097a9e8-f019-48d8-8c20-4804afb6b16d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:16.165507 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097a9e8-f019-48d8-8c20-4804afb6b16d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:16.192665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1463/2922] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:59:16 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:16.197740 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f275f37-1c05-485a-b096-c7379fd9bf25 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:16.210549 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f275f37-1c05-485a-b096-c7379fd9bf25 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:16.217220 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588173e6-9a06-4ea2-a3d9-ea4e1af53a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:16.217905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588173e6-9a06-4ea2-a3d9-ea4e1af53a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:16.218191 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588173e6-9a06-4ea2-a3d9-ea4e1af53a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:16.218458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588173e6-9a06-4ea2-a3d9-ea4e1af53a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:16.219133 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588173e6-9a06-4ea2-a3d9-ea4e1af53a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:16.221206 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588173e6-9a06-4ea2-a3d9-ea4e1af53a3f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:16.235005 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1460/2923] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:16.240354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097a9e8-f019-48d8-8c20-4804afb6b16d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:16.246242 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:16.254919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097a9e8-f019-48d8-8c20-4804afb6b16d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:16.263335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fe78d-e6c3-4a34-a3ea-381c0986fc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:16.264506 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fe78d-e6c3-4a34-a3ea-381c0986fc40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:16.265039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fe78d-e6c3-4a34-a3ea-381c0986fc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:16.265518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fe78d-e6c3-4a34-a3ea-381c0986fc40 admin admin] PATH_INFO: `/v3/projects/340e36f4590f426aaca0776281f540d3/users/d1c77eba159e4cfd90a559a44c70bc01/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:16.293870 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fe78d-e6c3-4a34-a3ea-381c0986fc40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340e36f4590f426aaca0776281f540d3, user_id=d1c77eba159e4cfd90a559a44c70bc01, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:16.299326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fe78d-e6c3-4a34-a3ea-381c0986fc40 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:16.323262 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1464/2924] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:59:16 2022] PUT /identity/v3/projects/340e36f4590f426aaca0776281f540d3/users/d1c77eba159e4cfd90a559a44c70bc01/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:16.328223 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588173e6-9a06-4ea2-a3d9-ea4e1af53a3f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:16.341042 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588173e6-9a06-4ea2-a3d9-ea4e1af53a3f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:16.368381 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:16.422135 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba8a53-e7e7-4ec7-9de8-20c07fc8e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:16.423103 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba8a53-e7e7-4ec7-9de8-20c07fc8e93a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:16.423468 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba8a53-e7e7-4ec7-9de8-20c07fc8e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:16.423786 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba8a53-e7e7-4ec7-9de8-20c07fc8e93a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:16.424504 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba8a53-e7e7-4ec7-9de8-20c07fc8e93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:16.426382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba8a53-e7e7-4ec7-9de8-20c07fc8e93a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:16.438662 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1461/2925] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 12:59:16.443933 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fe78d-e6c3-4a34-a3ea-381c0986fc40 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:16.455320 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fe78d-e6c3-4a34-a3ea-381c0986fc40 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:16.461265 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:16.461983 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:16.462252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:16.462525 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] PATH_INFO: `/v3/projects/340e36f4590f426aaca0776281f540d3/users/d1c77eba159e4cfd90a559a44c70bc01/roles/ce6acb63b60a4f53a9ff9a32054b9b63` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:16.464843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=340e36f4590f426aaca0776281f540d3, user_id=d1c77eba159e4cfd90a559a44c70bc01, role_id=ce6acb63b60a4f53a9ff9a32054b9b63)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:16.467766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:16.480200 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1465/2926] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:59:16 2022] PUT /identity/v3/projects/340e36f4590f426aaca0776281f540d3/users/d1c77eba159e4cfd90a559a44c70bc01/roles/ce6acb63b60a4f53a9ff9a32054b9b63 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:16.494576 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b828c1-10f7-4cf5-9c00-7b984e0b3472 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:16.495098 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b828c1-10f7-4cf5-9c00-7b984e0b3472 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:16.496526 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b828c1-10f7-4cf5-9c00-7b984e0b3472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:16.550284 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d6b828c1-10f7-4cf5-9c00-7b984e0b3472 None None] MFA Rules not processed for user `d1c77eba159e4cfd90a559a44c70bc01`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:16.552299 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d6b828c1-10f7-4cf5-9c00-7b984e0b3472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:16.647826 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1462/2927] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:16.664163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:16.666842 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:16.753984 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081039d-ec2b-4deb-b676-5bb0381931ba admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:16.758641 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d7bd6-521e-4cca-a8da-efe9d7c578c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:16.759273 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7bd6-521e-4cca-a8da-efe9d7c578c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:16.759562 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7bd6-521e-4cca-a8da-efe9d7c578c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:16.759832 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7bd6-521e-4cca-a8da-efe9d7c578c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:16.771377 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d7bd6-521e-4cca-a8da-efe9d7c578c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:16.772866 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d7bd6-521e-4cca-a8da-efe9d7c578c7 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:16.783558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1466/2928] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:59:17.301855 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b828c1-10f7-4cf5-9c00-7b984e0b3472 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:17.378614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b828c1-10f7-4cf5-9c00-7b984e0b3472 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:17.386376 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:17.387424 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:17.387932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:17.388422 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] PATH_INFO: `/v3/users/7668128b22af442dac5a9faac23c6ca5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:17.404395 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=7668128b22af442dac5a9faac23c6ca5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:17.407957 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:17.467817 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7668128b22af442dac5a9faac23c6ca5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:17.468591 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7668128b22af442dac5a9faac23c6ca5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:17.470202 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:17.485975 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7668128b22af442dac5a9faac23c6ca5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:17.495086 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7668128b22af442dac5a9faac23c6ca5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:17.506880 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7668128b22af442dac5a9faac23c6ca5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:17.514111 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1463/2929] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:59:17 2022] DELETE /identity/v3/users/7668128b22af442dac5a9faac23c6ca5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:17.519435 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f62e6-8b33-4668-b939-bf73038c132f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:17.520270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f62e6-8b33-4668-b939-bf73038c132f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:17.520270 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f62e6-8b33-4668-b939-bf73038c132f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:17.541850 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-db0f62e6-8b33-4668-b939-bf73038c132f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:17.543109 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-db0f62e6-8b33-4668-b939-bf73038c132f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:17.619929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1467/2930] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:17.631403 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:17.633070 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:17.706304 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075e025-f371-4303-9be0-b5d8cb7075ab admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:17.710986 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a595046-2c0f-4860-82c7-67a401148246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:17.711656 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a595046-2c0f-4860-82c7-67a401148246 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:17.711932 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a595046-2c0f-4860-82c7-67a401148246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:17.712201 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a595046-2c0f-4860-82c7-67a401148246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:17.722958 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a595046-2c0f-4860-82c7-67a401148246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:17.725176 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a595046-2c0f-4860-82c7-67a401148246 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:17.736488 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1464/2931] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 12:59:18.084136 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f62e6-8b33-4668-b939-bf73038c132f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:18.086079 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-db0f62e6-8b33-4668-b939-bf73038c132f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:18.101073 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f62e6-8b33-4668-b939-bf73038c132f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:18.110447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:18.111204 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:18.111722 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:18.112112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] PATH_INFO: `/v3/projects/b25b02a1341042f8aeaf38e4f6006315` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:18.114139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25b02a1341042f8aeaf38e4f6006315)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:18.117485 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:18.200622 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25b02a1341042f8aeaf38e4f6006315', 'request_id': 'req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.212213 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25b02a1341042f8aeaf38e4f6006315', 'request_id': 'req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.215438 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:18.226349 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25b02a1341042f8aeaf38e4f6006315', 'request_id': 'req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.248494 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] The token cache is being invalidate because project b25b02a1341042f8aeaf38e4f6006315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:59:18.249058 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.250072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.252956 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1468/2932] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:59:18 2022] DELETE /identity/v3/projects/b25b02a1341042f8aeaf38e4f6006315 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:18.257787 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a595046-2c0f-4860-82c7-67a401148246 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:18.269208 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a595046-2c0f-4860-82c7-67a401148246 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:18.337913 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:18.338571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:18.338847 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:18.339122 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] PATH_INFO: `/v3/users/d1c77eba159e4cfd90a559a44c70bc01` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:18.340363 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c77eba159e4cfd90a559a44c70bc01)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:18.342337 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:18.422528 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c77eba159e4cfd90a559a44c70bc01'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.423738 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c77eba159e4cfd90a559a44c70bc01'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.445574 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c77eba159e4cfd90a559a44c70bc01'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.454670 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:18.460626 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c77eba159e4cfd90a559a44c70bc01'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.472051 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffb64156-01ab-4050-af27-f11d133e92f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c77eba159e4cfd90a559a44c70bc01'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:18.479411 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1465/2933] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:59:18 2022] DELETE /identity/v3/users/d1c77eba159e4cfd90a559a44c70bc01 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:18.877853 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:18.881433 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:18.916931 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82d8cf-68f8-45d6-a69f-0d27d5d8d180 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:19.012026 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:19.012809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.013185 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.013548 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] PATH_INFO: `/v3/projects/340e36f4590f426aaca0776281f540d3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.015505 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=340e36f4590f426aaca0776281f540d3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:19.017679 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:19.078533 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340e36f4590f426aaca0776281f540d3', 'request_id': 'req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:19.085982 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340e36f4590f426aaca0776281f540d3', 'request_id': 'req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:19.096463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340e36f4590f426aaca0776281f540d3', 'request_id': 'req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:19.110631 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] The token cache is being invalidate because project 340e36f4590f426aaca0776281f540d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:59:19.110977 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:19.111843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ed5248a-da9e-453f-a0a6-e8a57e5776c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:19.113466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1469/2934] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:59:18 2022] DELETE /identity/v3/projects/340e36f4590f426aaca0776281f540d3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:19.121654 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be9fd-444d-4a80-b6f6-248ac6a93102 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.121945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be9fd-444d-4a80-b6f6-248ac6a93102 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.122203 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be9fd-444d-4a80-b6f6-248ac6a93102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.146406 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-716be9fd-444d-4a80-b6f6-248ac6a93102 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:19.147717 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-716be9fd-444d-4a80-b6f6-248ac6a93102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:19.221100 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1466/2935] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:19.229446 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f964e6-1977-40e8-aa4f-aa1cbc5fb339 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.229729 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f964e6-1977-40e8-aa4f-aa1cbc5fb339 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.229998 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f964e6-1977-40e8-aa4f-aa1cbc5fb339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.252673 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-50f964e6-1977-40e8-aa4f-aa1cbc5fb339 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:19.253965 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-50f964e6-1977-40e8-aa4f-aa1cbc5fb339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:19.264795 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1470/2936] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:19.269557 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be9fd-444d-4a80-b6f6-248ac6a93102 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:19.280691 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be9fd-444d-4a80-b6f6-248ac6a93102 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:19.286288 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:19.287008 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.287372 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.287714 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.288425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:59:19.288762 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:19.290736 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:19.312624 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1467/2937] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:59:19.318153 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f964e6-1977-40e8-aa4f-aa1cbc5fb339 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:19.335608 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f964e6-1977-40e8-aa4f-aa1cbc5fb339 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:19.344139 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70765c4c-7525-41b8-a13e-b1895c4a9822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:19.345127 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70765c4c-7525-41b8-a13e-b1895c4a9822 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.345552 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70765c4c-7525-41b8-a13e-b1895c4a9822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.346019 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70765c4c-7525-41b8-a13e-b1895c4a9822 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.347203 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70765c4c-7525-41b8-a13e-b1895c4a9822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:19.351074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70765c4c-7525-41b8-a13e-b1895c4a9822 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:19.401045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1471/2938] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:59:19 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:19.404735 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:19.423905 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7113e0d-a544-445a-8c33-7fbcdceb7421 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:19.529899 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad2203-69cf-4790-aa9f-26bf087cfd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:19.530849 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2203-69cf-4790-aa9f-26bf087cfd3e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.531267 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2203-69cf-4790-aa9f-26bf087cfd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.531743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2203-69cf-4790-aa9f-26bf087cfd3e admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.532879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad2203-69cf-4790-aa9f-26bf087cfd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:19.537035 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad2203-69cf-4790-aa9f-26bf087cfd3e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:19.568444 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1468/2939] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:59:19 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:19.575061 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70765c4c-7525-41b8-a13e-b1895c4a9822 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:19.592112 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70765c4c-7525-41b8-a13e-b1895c4a9822 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:19.598386 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d7ea5-77bc-4545-a2bd-88de7a1b736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:19.599035 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d7ea5-77bc-4545-a2bd-88de7a1b736f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.599356 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d7ea5-77bc-4545-a2bd-88de7a1b736f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.599634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d7ea5-77bc-4545-a2bd-88de7a1b736f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.600499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d7ea5-77bc-4545-a2bd-88de7a1b736f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:19.602198 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d7ea5-77bc-4545-a2bd-88de7a1b736f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:19.615955 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1472/2940] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:19.620345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad2203-69cf-4790-aa9f-26bf087cfd3e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:19.632593 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad2203-69cf-4790-aa9f-26bf087cfd3e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:19.638518 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b76e-e011-40d1-93b2-cfa38354ee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:19.639154 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b76e-e011-40d1-93b2-cfa38354ee32 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.639469 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b76e-e011-40d1-93b2-cfa38354ee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.639747 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b76e-e011-40d1-93b2-cfa38354ee32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.640417 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655b76e-e011-40d1-93b2-cfa38354ee32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:19.642108 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655b76e-e011-40d1-93b2-cfa38354ee32 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:19.660333 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1469/2941] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:19.667041 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d7ea5-77bc-4545-a2bd-88de7a1b736f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:19.686051 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d7ea5-77bc-4545-a2bd-88de7a1b736f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:19.692335 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:19.693090 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.693370 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.693730 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] PATH_INFO: `/v3/projects/7d1719f29f5f447188162e793f2df258/users/309e2851d6b245a18ad7afe0b098d671/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.719298 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1719f29f5f447188162e793f2df258, user_id=309e2851d6b245a18ad7afe0b098d671, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:19.724806 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:19.744715 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1473/2942] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:59:19 2022] PUT /identity/v3/projects/7d1719f29f5f447188162e793f2df258/users/309e2851d6b245a18ad7afe0b098d671/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:19.755960 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d05ced-f17a-4d3d-b12e-a849c0fb260a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.756382 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d05ced-f17a-4d3d-b12e-a849c0fb260a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.756743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d05ced-f17a-4d3d-b12e-a849c0fb260a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.793581 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-17d05ced-f17a-4d3d-b12e-a849c0fb260a None None] MFA Rules not processed for user `309e2851d6b245a18ad7afe0b098d671`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:19.794793 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-17d05ced-f17a-4d3d-b12e-a849c0fb260a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:19.848374 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:19.884929 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1470/2943] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:19.902239 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:19.904161 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:19.916534 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:19.977879 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af252b4-731d-4fb7-9fa6-dc42b9464e82 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:19.981769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ffaff-543e-4175-9ed9-e08d86ee545a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef756349e5844e59d7b20072f63c41c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:19.982414 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ffaff-543e-4175-9ed9-e08d86ee545a service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:19.982696 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ffaff-543e-4175-9ed9-e08d86ee545a service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:19.982968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ffaff-543e-4175-9ed9-e08d86ee545a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:19.992775 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ffaff-543e-4175-9ed9-e08d86ee545a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:19.994260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ffaff-543e-4175-9ed9-e08d86ee545a service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:20.005751 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1474/2944] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:59:29.881477 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=8 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:30.000317 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=22 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:48.545776 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d05ced-f17a-4d3d-b12e-a849c0fb260a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:48.619054 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d05ced-f17a-4d3d-b12e-a849c0fb260a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:48.624568 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:48.625302 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:48.625651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:48.626026 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] PATH_INFO: `/v3/users/309e2851d6b245a18ad7afe0b098d671` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:48.627579 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=309e2851d6b245a18ad7afe0b098d671)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:48.629598 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:48.686128 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309e2851d6b245a18ad7afe0b098d671'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:48.686858 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309e2851d6b245a18ad7afe0b098d671'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:48.700278 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309e2851d6b245a18ad7afe0b098d671'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:48.708651 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309e2851d6b245a18ad7afe0b098d671'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:48.718876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309e2851d6b245a18ad7afe0b098d671'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:48.726850 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1471/2945] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 12:59:48 2022] DELETE /identity/v3/users/309e2851d6b245a18ad7afe0b098d671 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:48.732894 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9bc2-754b-49be-964c-e352023b16a5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:48.733478 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9bc2-754b-49be-964c-e352023b16a5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:48.734229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9bc2-754b-49be-964c-e352023b16a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:48.768727 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-940f9bc2-754b-49be-964c-e352023b16a5 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:48.770816 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-940f9bc2-754b-49be-964c-e352023b16a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:48.849280 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1475/2946] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:48.860685 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:48.862916 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:48.939370 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d9025-7473-45d8-b09c-b55353b70263 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:48.943448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fdbe7-29b9-487f-a994-8909cd4b7bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:48.944131 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fdbe7-29b9-487f-a994-8909cd4b7bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:48.944502 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fdbe7-29b9-487f-a994-8909cd4b7bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:48.944818 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fdbe7-29b9-487f-a994-8909cd4b7bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:48.957268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fdbe7-29b9-487f-a994-8909cd4b7bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:48.958791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fdbe7-29b9-487f-a994-8909cd4b7bb8 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:48.978454 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1472/2947] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 12:59:49.339769 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9bc2-754b-49be-964c-e352023b16a5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:49.341744 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-940f9bc2-754b-49be-964c-e352023b16a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:49.357176 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9bc2-754b-49be-964c-e352023b16a5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:49.365163 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:49.368544 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:49.369103 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:49.369584 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] PATH_INFO: `/v3/projects/7d1719f29f5f447188162e793f2df258` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:49.371907 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1719f29f5f447188162e793f2df258)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:49.375820 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:49.441986 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1719f29f5f447188162e793f2df258', 'request_id': 'req-d5f025e2-abb4-4042-9760-b18d4de543da'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:49.450275 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1719f29f5f447188162e793f2df258', 'request_id': 'req-d5f025e2-abb4-4042-9760-b18d4de543da'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:49.461030 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1719f29f5f447188162e793f2df258', 'request_id': 'req-d5f025e2-abb4-4042-9760-b18d4de543da'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:49.475205 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] The token cache is being invalidate because project 7d1719f29f5f447188162e793f2df258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 12:59:49.475595 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:49.476324 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d5f025e2-abb4-4042-9760-b18d4de543da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 12:59:49.478000 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1476/2948] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 12:59:49 2022] DELETE /identity/v3/projects/7d1719f29f5f447188162e793f2df258 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:49.483449 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378c46e-02aa-4774-ada9-d93e3130e62b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:49.483876 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378c46e-02aa-4774-ada9-d93e3130e62b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:49.484189 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378c46e-02aa-4774-ada9-d93e3130e62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:49.506459 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5378c46e-02aa-4774-ada9-d93e3130e62b None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:49.507737 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5378c46e-02aa-4774-ada9-d93e3130e62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:49.537225 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:49.602791 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1473/2949] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 12:59:49.612072 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b1e49-9198-4660-8c2e-70f39a465e6f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:49.612557 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b1e49-9198-4660-8c2e-70f39a465e6f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:49.612734 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b1e49-9198-4660-8c2e-70f39a465e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:49.638291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b22b1e49-9198-4660-8c2e-70f39a465e6f None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:49.639758 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b22b1e49-9198-4660-8c2e-70f39a465e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:49.649391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1477/2950] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:49.654512 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378c46e-02aa-4774-ada9-d93e3130e62b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:49.674213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378c46e-02aa-4774-ada9-d93e3130e62b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:49.680681 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:49.681429 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:49.681810 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:49.682170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:49.682946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 12:59:49.683399 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:49.685216 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:49.701493 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1474/2951] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 31 12:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 12:59:49.706538 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b1e49-9198-4660-8c2e-70f39a465e6f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:49.717372 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b1e49-9198-4660-8c2e-70f39a465e6f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:49.724508 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d8a74-9c4d-4048-aebc-ccfe6a31a19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:49.725181 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d8a74-9c4d-4048-aebc-ccfe6a31a19c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:49.725499 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d8a74-9c4d-4048-aebc-ccfe6a31a19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:49.725809 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d8a74-9c4d-4048-aebc-ccfe6a31a19c admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:49.726783 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d8a74-9c4d-4048-aebc-ccfe6a31a19c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:49.729248 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d8a74-9c4d-4048-aebc-ccfe6a31a19c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:49.764501 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1478/2952] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 31 12:59:49 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:49.770562 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:49.782790 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853113-155c-436e-a491-264d24ca4ba2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:49.857426 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7f2ab-5d36-486f-b1b7-02b42cf9919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:49.858157 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f2ab-5d36-486f-b1b7-02b42cf9919c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:49.858478 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f2ab-5d36-486f-b1b7-02b42cf9919c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:49.858795 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f2ab-5d36-486f-b1b7-02b42cf9919c admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:49.859560 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7f2ab-5d36-486f-b1b7-02b42cf9919c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:49.861572 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7f2ab-5d36-486f-b1b7-02b42cf9919c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:49.897261 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1475/2953] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 31 12:59:49 2022] POST /identity/v3/users => generated 282 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 12:59:49.902749 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d8a74-9c4d-4048-aebc-ccfe6a31a19c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:49.915724 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d8a74-9c4d-4048-aebc-ccfe6a31a19c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:49.920855 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a1d01-c8bb-4ecb-b199-97a0613c9a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:49.921523 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a1d01-c8bb-4ecb-b199-97a0613c9a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:49.921800 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a1d01-c8bb-4ecb-b199-97a0613c9a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:49.922069 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a1d01-c8bb-4ecb-b199-97a0613c9a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:49.922728 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a1d01-c8bb-4ecb-b199-97a0613c9a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:49.924415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a1d01-c8bb-4ecb-b199-97a0613c9a14 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:49.938011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1479/2954] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:49.942884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7f2ab-5d36-486f-b1b7-02b42cf9919c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:49.955709 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7f2ab-5d36-486f-b1b7-02b42cf9919c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:49.961813 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd216a-d596-494a-9817-ecf5cb9c2166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:49.962750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd216a-d596-494a-9817-ecf5cb9c2166 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:49.963071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd216a-d596-494a-9817-ecf5cb9c2166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:49.963436 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd216a-d596-494a-9817-ecf5cb9c2166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:49.964185 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd216a-d596-494a-9817-ecf5cb9c2166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:49.966253 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd216a-d596-494a-9817-ecf5cb9c2166 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:49.980217 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1476/2955] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 31 12:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 12:59:49.985302 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a1d01-c8bb-4ecb-b199-97a0613c9a14 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:49.999487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a1d01-c8bb-4ecb-b199-97a0613c9a14 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:50.008011 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:50.009119 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:50.009581 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:50.010045 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] PATH_INFO: `/v3/projects/0016b67a2e844ff393d07373b87e2177/users/26584f3499f8477586d52d7e64296d43/roles/07097b9e79ae4ff1bf60ddef2b681c51` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:50.036125 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0016b67a2e844ff393d07373b87e2177, user_id=26584f3499f8477586d52d7e64296d43, role_id=07097b9e79ae4ff1bf60ddef2b681c51)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:50.039229 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:50.061130 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1480/2956] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 31 12:59:49 2022] PUT /identity/v3/projects/0016b67a2e844ff393d07373b87e2177/users/26584f3499f8477586d52d7e64296d43/roles/07097b9e79ae4ff1bf60ddef2b681c51 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 12:59:50.074945 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03f8fb-859f-4c8b-9f80-14f6866c1cf7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:50.075452 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03f8fb-859f-4c8b-9f80-14f6866c1cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:50.075904 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03f8fb-859f-4c8b-9f80-14f6866c1cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:50.115741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dd03f8fb-859f-4c8b-9f80-14f6866c1cf7 None None] MFA Rules not processed for user `26584f3499f8477586d52d7e64296d43`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 12:59:50.117199 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dd03f8fb-859f-4c8b-9f80-14f6866c1cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:50.199439 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1477/2957] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 12:59:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 12:59:50.211906 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 12:59:50.213911 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 12:59:50.240526 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 12:59:50.289558 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b6cb7-c039-497e-90ac-f6b00cd03f74 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 12:59:50.293457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfef0d-6672-43ca-baf7-f6fe8527646b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 12:59:50.294134 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfef0d-6672-43ca-baf7-f6fe8527646b service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 12:59:50.294454 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfef0d-6672-43ca-baf7-f6fe8527646b service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 12:59:50.294813 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfef0d-6672-43ca-baf7-f6fe8527646b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 12:59:50.305463 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfef0d-6672-43ca-baf7-f6fe8527646b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 12:59:50.306979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfef0d-6672-43ca-baf7-f6fe8527646b service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 12:59:50.318071 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1481/2958] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 12:59:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 13:00:00.195556 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:00:00.313518 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=17 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:00:06.152094 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03f8fb-859f-4c8b-9f80-14f6866c1cf7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:06.153604 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dd03f8fb-859f-4c8b-9f80-14f6866c1cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:06.171462 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03f8fb-859f-4c8b-9f80-14f6866c1cf7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:06.177448 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3393d3cb0e4bbcb21e403c431bf61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:06.178463 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:06.179078 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:06.179640 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:06.182910 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:06.268509 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:06.270315 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:06.281268 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1478/2959] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:00:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:00:14.562466 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74963-a7ae-4356-ad38-7fb3af891543 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:14.563646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74963-a7ae-4356-ad38-7fb3af891543 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:14.563646 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef74963-a7ae-4356-ad38-7fb3af891543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:14.588838 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4ef74963-a7ae-4356-ad38-7fb3af891543 None None] MFA Rules not processed for user `171b9c24597545a1a5c8388849ef5819`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 13:00:14.590106 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ef74963-a7ae-4356-ad38-7fb3af891543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:14.599156 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1482/2960] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 13:00:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 13:00:15.022631 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:15.024404 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:15.099566 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b7216-a19f-4208-b474-c53363e398fe service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:15.103809 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:15.104499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:15.104815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:15.105153 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:15.107330 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:15.119112 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:15.120692 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:15.141538 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1479/2961] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:00:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:00:16.759665 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef74963-a7ae-4356-ad38-7fb3af891543 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:16.761777 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ef74963-a7ae-4356-ad38-7fb3af891543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:16.860365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef74963-a7ae-4356-ad38-7fb3af891543 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:16.870226 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:16.871313 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:16.871803 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:16.872295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:16.875441 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:16.896170 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:16.897968 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:16.913636 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1483/2962] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:00:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:00:23.476231 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:23.478732 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:23.579973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1368f-415b-49d9-81a8-ff77e0b00dbe service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:23.589241 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a2ea9-4675-416e-993c-6b73c13eab93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:23.589896 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2ea9-4675-416e-993c-6b73c13eab93 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:23.590177 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2ea9-4675-416e-993c-6b73c13eab93 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:23.590571 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2ea9-4675-416e-993c-6b73c13eab93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:23.592739 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-261a2ea9-4675-416e-993c-6b73c13eab93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:23.607987 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a2ea9-4675-416e-993c-6b73c13eab93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:23.609570 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a2ea9-4675-416e-993c-6b73c13eab93 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:23.622517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1480/2963] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:00:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:00:26.908617 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=38 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:00:32.579896 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:32.668979 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11e103-04c2-452b-8d5a-05c95e216407 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:32.674291 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:32.675080 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:32.675517 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:32.675916 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] PATH_INFO: `/v3/users/26584f3499f8477586d52d7e64296d43` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:32.678281 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26584f3499f8477586d52d7e64296d43)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:32.680385 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:32.742315 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26584f3499f8477586d52d7e64296d43'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:32.742912 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26584f3499f8477586d52d7e64296d43'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:32.753018 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26584f3499f8477586d52d7e64296d43'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:32.754014 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26584f3499f8477586d52d7e64296d43'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:32.771050 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26584f3499f8477586d52d7e64296d43'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:32.787667 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1484/2964] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 13:00:32 2022] DELETE /identity/v3/users/26584f3499f8477586d52d7e64296d43 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 13:00:32.792950 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538ba9-363a-4e83-a4a1-34893ac65ba7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:32.793550 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538ba9-363a-4e83-a4a1-34893ac65ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:32.793665 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538ba9-363a-4e83-a4a1-34893ac65ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:32.802908 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:00:32.820362 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-49538ba9-363a-4e83-a4a1-34893ac65ba7 None None] MFA Rules not processed for user `186838d97c1140bb9bd1a7296d1414bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 13:00:32.821667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-49538ba9-363a-4e83-a4a1-34893ac65ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:32.901071 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1481/2965] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 31 13:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 13:00:32.915088 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:32.917252 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:32.996657 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efbc74-e273-45cf-8f26-9691b09dedd0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:33.001928 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:33.002691 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:33.003038 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:33.003351 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:33.018939 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:33.021919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:33.042241 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1485/2966] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:00:33.408688 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538ba9-363a-4e83-a4a1-34893ac65ba7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:33.411937 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-49538ba9-363a-4e83-a4a1-34893ac65ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:33.426820 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538ba9-363a-4e83-a4a1-34893ac65ba7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:33.434445 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:33.435700 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:33.436170 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:33.436659 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] PATH_INFO: `/v3/projects/0016b67a2e844ff393d07373b87e2177` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:33.438999 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0016b67a2e844ff393d07373b87e2177)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:33.443146 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:33.513391 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0016b67a2e844ff393d07373b87e2177', 'request_id': 'req-ce5671b3-d119-40a2-9c52-59ce17593d05'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:33.524696 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0016b67a2e844ff393d07373b87e2177', 'request_id': 'req-ce5671b3-d119-40a2-9c52-59ce17593d05'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:33.536815 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0016b67a2e844ff393d07373b87e2177', 'request_id': 'req-ce5671b3-d119-40a2-9c52-59ce17593d05'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:33.553061 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] The token cache is being invalidate because project 0016b67a2e844ff393d07373b87e2177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 13:00:33.553409 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:33.554219 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:00:33.555831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1482/2967] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 13:00:33 2022] DELETE /identity/v3/projects/0016b67a2e844ff393d07373b87e2177 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 13:00:35.075919 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:35.077793 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:35.115843 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:00:35.200432 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae99e0d-6f37-4ea3-baef-e8355f613087 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:35.278177 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:35.278950 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:35.279305 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:35.279643 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:35.281910 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:35.296602 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:35.299364 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:35.312326 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1486/2968] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:00:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 13:00:40.250313 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:00:40.252471 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:40.274327 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5671b3-d119-40a2-9c52-59ce17593d05 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:00:40.281132 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b113316bc3348b5a12c1d7b616bf4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:00:40.281867 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:00:40.282174 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:00:40.282443 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:00:40.284743 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:00:40.299956 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:00:40.301614 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:00:40.315064 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1483/2969] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:00:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:00:45.306670 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=52 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:00:50.309765 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:01:05.875629 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:01:05.877278 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:01:05.963322 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d577-fa28-4999-8a48-daec4f1e8433 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:01:05.967487 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:01:05.968118 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:01:05.968403 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:01:05.968671 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:01:05.970658 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:01:05.982451 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:01:05.983937 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:01:05.994601 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1487/2970] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:01:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:01:15.989105 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=30 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:02:24.268125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:02:24.271877 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:24.421667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49627e-c369-4f82-ab58-673764322efa service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:02:24.431695 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:02:24.432884 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:02:24.433398 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:02:24.433993 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:02:24.437481 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:24.460139 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:02:24.462230 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:02:24.476733 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1484/2971] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:02:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:02:34.469277 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=32 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:02:37.550903 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:02:37.552520 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:37.568766 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75324f-cc20-471b-991d-dff85dd39fe6 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:02:37.576929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8c7348ad2e457e80e1fdd2d93aacc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:02:37.576929 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:02:37.577988 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:02:37.578085 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:02:37.581199 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:37.663785 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:02:37.665243 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:02:37.676719 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1488/2972] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:02:41.632492 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:02:41.634643 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:41.649667 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf83616-e06f-4fb3-8ca4-39a407d95801 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:02:41.656050 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:02:41.656831 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:02:41.657213 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:02:41.657575 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:02:41.660125 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:41.675345 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:02:41.677431 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:02:41.693515 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1485/2973] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:02:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:02:45.762148 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:02:45.783095 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf5d4b-4013-468a-85ff-2be86bd811c3 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:02:45.793354 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:02:45.794457 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:02:45.795634 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:02:45.796044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:02:45.798682 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:45.815295 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:02:45.817274 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:02:45.834512 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1489/2974] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:02:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 13:02:45.852750 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:02:45.855458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:45.877930 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05970ba-90c6-4f3f-acb4-7c57fff02dae service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:02:45.891342 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:02:45.892366 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:02:45.892803 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:02:45.893222 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] PATH_INFO: `/v3/users/171b9c24597545a1a5c8388849ef5819` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:02:45.895517 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=171b9c24597545a1a5c8388849ef5819)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:02:45.899352 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:02:45.960197 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '171b9c24597545a1a5c8388849ef5819'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:45.960934 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '171b9c24597545a1a5c8388849ef5819'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:45.976458 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '171b9c24597545a1a5c8388849ef5819'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:45.985109 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '171b9c24597545a1a5c8388849ef5819'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:45.994927 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '171b9c24597545a1a5c8388849ef5819'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.005323 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1486/2975] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 13:02:45 2022] DELETE /identity/v3/users/171b9c24597545a1a5c8388849ef5819 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 13:02:46.234872 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:02:46.253074 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da3caa-26ea-4015-ae53-05d48719cec2 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:02:46.352771 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:02:46.353652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:02:46.354044 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:02:46.354396 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] PATH_INFO: `/v3/projects/b5e2d3e145e04936bf60fe93b5011cc2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:02:46.356010 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5e2d3e145e04936bf60fe93b5011cc2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:02:46.358089 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:02:46.431819 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5e2d3e145e04936bf60fe93b5011cc2', 'request_id': 'req-de0e4337-b9bc-48f4-8ae3-c781dfed8183'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.443004 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5e2d3e145e04936bf60fe93b5011cc2', 'request_id': 'req-de0e4337-b9bc-48f4-8ae3-c781dfed8183'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.455415 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5e2d3e145e04936bf60fe93b5011cc2', 'request_id': 'req-de0e4337-b9bc-48f4-8ae3-c781dfed8183'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.469747 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] The token cache is being invalidate because project b5e2d3e145e04936bf60fe93b5011cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 13:02:46.470028 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.470739 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.472365 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1490/2976] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 13:02:46 2022] DELETE /identity/v3/projects/b5e2d3e145e04936bf60fe93b5011cc2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 13:02:46.477781 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:02:46.479499 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:46.495719 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a06255-e0ee-426c-ba36-4ad627ddc866 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:02:46.592121 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:02:46.593653 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:02:46.594029 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:02:46.594410 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] PATH_INFO: `/v3/users/7c36cbb15c5c49d0a1ea96c6cc0b9ab8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:02:46.596711 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c36cbb15c5c49d0a1ea96c6cc0b9ab8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:02:46.599646 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:02:46.663973 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c36cbb15c5c49d0a1ea96c6cc0b9ab8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.664373 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c36cbb15c5c49d0a1ea96c6cc0b9ab8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.679588 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:02:46.682885 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c36cbb15c5c49d0a1ea96c6cc0b9ab8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.698741 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c36cbb15c5c49d0a1ea96c6cc0b9ab8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.712473 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c36cbb15c5c49d0a1ea96c6cc0b9ab8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:46.720425 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1487/2977] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 31 13:02:46 2022] DELETE /identity/v3/users/7c36cbb15c5c49d0a1ea96c6cc0b9ab8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 13:02:46.961644 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:02:46.963639 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:02:46.984652 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4337-b9bc-48f4-8ae3-c781dfed8183 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:02:47.002096 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:02:47.073447 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '186838d97c1140bb9bd1a7296d1414bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6117aaa4eb34295b2216e51a40eacde', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:02:47.074039 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:02:47.074312 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:02:47.074583 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] PATH_INFO: `/v3/projects/63297fdcd97b4e2181d3d2fcde70c516` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:02:47.075924 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63297fdcd97b4e2181d3d2fcde70c516)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:02:47.077887 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:02:47.135605 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63297fdcd97b4e2181d3d2fcde70c516', 'request_id': 'req-83b3004c-e286-45b4-859a-0395e4e3d612'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:47.143391 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63297fdcd97b4e2181d3d2fcde70c516', 'request_id': 'req-83b3004c-e286-45b4-859a-0395e4e3d612'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:47.154810 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63297fdcd97b4e2181d3d2fcde70c516', 'request_id': 'req-83b3004c-e286-45b4-859a-0395e4e3d612'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:47.168253 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] The token cache is being invalidate because project 63297fdcd97b4e2181d3d2fcde70c516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 13:02:47.168565 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:47.169260 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-83b3004c-e286-45b4-859a-0395e4e3d612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 13:02:47.170852 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1491/2978] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 31 13:02:46 2022] DELETE /identity/v3/projects/63297fdcd97b4e2181d3d2fcde70c516 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 13:02:56.715801 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:02:57.165084 dv-cinder-855359-1-iscsi devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 13:07:25.954946 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 13:07:25.957489 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:07:26.229860 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc099118-7ff8-47dd-8fc1-f80e78dd7935 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 13:07:26.365728 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e66b4b-4009-4775-8e6d-6b614dc24b63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56361f445d894e73bae29764ae8b2b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c0eb399536f48f18191d97441cea577', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 13:07:26.367658 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e66b4b-4009-4775-8e6d-6b614dc24b63 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 13:07:26.368175 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e66b4b-4009-4775-8e6d-6b614dc24b63 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 13:07:26.368671 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e66b4b-4009-4775-8e6d-6b614dc24b63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 13:07:26.373622 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e1e66b4b-4009-4775-8e6d-6b614dc24b63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 13:07:26.398721 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e66b4b-4009-4775-8e6d-6b614dc24b63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 13:07:26.401889 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e66b4b-4009-4775-8e6d-6b614dc24b63 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 13:07:26.424879 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1488/2979] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 31 13:07:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 13:07:36.414978 dv-cinder-855359-1-iscsi devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=75 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}